Contemporary humans occupy the widest range of socioeconomic environments in their evolutionary history, and this has revealed unprecedented environmentally-induced plasticity in physical growth. This plasticity also has limits, and identifying those limits can help researchers: (1) parse when population differences arise from environmental inputs or not and (2) determine when it is possible to infer socioeconomic disparities from disparities in body form. To illustrate potential limits to environmental plasticity, we analyze body mass index (BMI) and height data from 1,768,962 women and 207,341 men (20–49 y) living in households exhibiting 1000-fold variation in household wealth (51 countries, 1985–2017, 164 surveys) across four world regions—sub-Saharan Africa, South Asia, Latin America, and North Africa and the Middle East. We find that relationships of environmental inputs with both mean height and BMI bottom out at roughly 100–700 USD per capita household wealth (2011 international units, PPP), but at different basal BMIs and basal heights for different regions. The relationship with resources tops out for BMI at around 20K–35K USD for women, with growth potential due to environmental inputs in the range of 6.2–8.4kg/m2. By contrast, mean BMI for men and mean height for both sexes remains sensitive to environmental inputs even at levels far above the low- and middle-income samples studied here. This suggest that further work integrating comparable data from low- and high-income samples should provide a better picture of the full range of environmental inputs on human height and BMI. We conclude by discussing how neglecting such population-specific limits to human growth can lead to erroneous inferences about population differences.