Surely only if you are inclined to worry. Rich people generally have fewer children than those living at subsistence level. To quote Mahatma Gandhi "the earth provides enough to satisfy everyman's need but not everyman's greed". The more we move towards equality the lower the replacement rate is seen to need to be and the population will start to go down. The more we appease the greedy the less likely this is to happen.