Does Increasing Sample Size Inevitably Lead to Statistical Significance? Insights From a Monte Carlo Simulation Study for Medical Research Design.

objectiveThis study addresses the common misconception that larger sample sizes inevitably produce smaller P-values and higher statistical power, using Monte Carlo simulations to examine the effect of sample size under scenarios with and without a true effect.

methodsMonte Carlo simulations were performed with 1000 iterations for each of 11 sample sizes per group (n = 5-800). Two scenarios were simulated: (1) both groups drawn from the same normal distribution (μ = 4.5, σ = 0.5; null hypothesis H

resultsWhen a true effect existed, increasing sample size led to a marked rise in statistical power, with the proportion of significant results increasing from 5.4% to 98.1%. In contrast, when no true effect existed, p-values remained uniformly distributed across all sample sizes, and the frequency of p < 0.05 consistently approximated the nominal 5% significance level. Importantly, increasing sample size under the null hypothesis did not increase the likelihood of detecting significance.

conclusionLarger sample sizes do not create statistical significance in the absence of a true effect; they only enhance the detection of effects that genuinely exist. Study design and statistical interpretation should therefore be guided by effect size and clinical relevance rather than by sample size alone.