ArticleNature human behaviour2020
Comparing meta-analyses and preregistered multiple-laboratory replication projects.
Article in Nature human behaviour, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 79 papers, 17 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
79 citing papers in PubMed, 17 syntheses or guidelines pooled it, 279 citations in OpenAlex.
- Heterogeneity in gender differences in self-reported political preferences, trust, and well-being across 39 European countries.Scientific reports · 2025Pooled it
- Impact of cardiovascular magnetic resonance-derived right ventricular ejection fraction on adverse outcomes: A robust Bayesian model-averaged meta-analysis.Journal of cardiovascular magnetic resonance : official journal of the Society for Cardiovascular Magnetic Resonance · 2024Pooled it
- Assessing the Evidential Value of Mental Fatigue and Exercise Research.Sports medicine (Auckland, N.Z.) · 2023Pooled it
- A systematic review of the strength of evidence for the most commonly recommended happiness strategies in mainstream media.Nature human behaviour · 2023Pooled it
- An umbrella review of randomized control trials on the effects of physical exercise on cognition.Nature human behaviour · 2023Pooled it
- Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies.Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc · 2023Pooled it
- Voice Patterns as Markers of Schizophrenia: Building a Cumulative Generalizable Approach Via a Cross-Linguistic and Meta-analysis Based Investigation.Schizophrenia bulletin · 2023Pooled it
- Pooled it
- Robust Bayesian meta-analysis: Model-averaging across complementary publication bias adjustment methods.Research synthesis methods · 2023Pooled it
- The Effect of Mindfulness-based Programs on Cognitive Function in Adults: A Systematic Review and Meta-analysis.Neuropsychology review · 2022Pooled it
- Temporal discounting across adulthood: A systematic review and meta-analysis.Psychology and aging · 2022Pooled it
- A meta-analytic investigation of the role of reward on inhibitory control.Quarterly journal of experimental psychology (2006) · 2021Pooled it
- Ecological impacts of water-based recreational activities on freshwater ecosystems: a global meta-analysis.Proceedings. Biological sciences · 2021Pooled it
- Can MRI measure myelin? Systematic review, qualitative assessment, and meta-analysis of studies validating microstructural imaging with myelin histology.NeuroImage · 2021Pooled it
- Using Monte Carlo experiments to select meta-analytic estimators.Research synthesis methods · 2021Pooled it
- Message Design Choices Don't Make Much Difference to Persuasiveness and Can't Be Counted On-Not Even When Moderating Conditions Are Specified.Frontiers in psychology · 2021Pooled it
- Is the Psychopathic Brain an Artifact of Coding Bias? A Systematic Review.Frontiers in psychology · 2021Pooled it
- Personalized prediction of response to three meditation practices: A randomized controlled trial.Behaviour research and therapy · 2026Trial
- ManyBabies 3: A Multi-Lab Study of Infant Algebraic Rule Learning.Developmental science · 2026Article
- How methodological cues shape trust in psychological research and legal judgements: a vignette study.Royal Society open science · 2026Article
19 more citing papers are in PubMed but not listed here.
Corrections and comments
- Erratum issued
Authors and funding
3 authors at 2 institutions in 2 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Many researchers rely on meta-analysis to summarize research evidence. However, there is a concern that publication bias and selective reporting may lead to biased meta-analytic effect sizes. We compare the results of meta-analyses to large-scale preregistered replications in psychology carried out at multiple laboratories. The multiple-laboratory replications provide precisely estimated effect sizes that do not suffer from publication bias or selective reporting. We searched the literature and identified 15 meta-analyses on the same topics as multiple-laboratory replications. We find that meta-analytic effect sizes are significantly different from replication effect sizes for 12 out of the 15 meta-replication pairs. These differences are systematic and, on average, meta-analytic effect sizes are almost three times as large as replication effect sizes. We also implement three methods of correcting meta-analysis for bias, but these methods do not substantively improve the meta-analytic results.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.