Meta-analyses of genome-wide association studies, which dominate genetic discovery, are based on data from diverse historical time periods and populations. Genetic scores derived from genome-wide association studies explain only a fraction of the heritability estimates obtained from whole-genome studies on single populations, known as the ‘hidden heritability’ puzzle. Using seven sampling populations (n = 35,062), we test whether hidden heritability is attributed to heterogeneity across sampling populations and time, showing that estimates are substantially smaller across populations compared with within populations. We show that the hidden heritability varies substantially: from zero for height to 20% for body mass index, 37% for education, 40% for age at first birth and up to 75% for number of children. Simulations demonstrate that our results are more likely to reflect heterogeneity in phenotypic measurement or gene–environment interactions than genetic heterogeneity. These findings have substantial implications for genetic discovery, suggesting that large homogenous datasets are required for behavioural phenotypes and that gene–environment interaction may be a central challenge for genetic discovery.