Is this selfish of me?

This is a great question. I struggled with the same thing because I'm the only guy in our family so felt pressure to 'carry on the family name.' I don't see it as selfish. Instead it's a sign of maturity and knowing what you want. If you did give in to the pressure and get married and have children you didn't really want, the children would know this. I've seen lots of situations where people have had children because that's what they felt they were 'supposed' to do. Now they resent them and don't care for them. That's a poor reason to bring children in to the world.