Wealthy people are often wealthy because they had capital to work with in the first place.

You’re flat wrong about this. Unless of course you’re comparing the middle class to the truly poor and calling them wealthy relatively.

You’re drawing the conclusion that wealthy people are fundamentally different (more willing to use leisure time actively because they’re just ‘go-getters) from average working people.

Well, they are. I’ve worked in fields that range from construction and day labor to, full service hospitality to start up information technology and yes, there is a difference between each of the groups of people that remain in these industries. Suggesting otherwise is to deny reality. There is a pervasive myth that everyone has equal abilities and those that don’t get rich or “succeed” were simply denied some opportunity along the way. That is utter nonsense. Of course there are outliers in each domain but averages don’t lie. Don’t trap yourself in the blank slate fallacy.