ArticlePloS one2020
Reproducibility of individual effect sizes in meta-analyses in psychology.
Article in PloS one, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 27 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
27 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Landscape of 2D Deep Learning Segmentation Networks Applied to CT Scan from Lung Cancer Patients: A Systematic Review.Journal of imaging informatics in medicine · 2025Pooled it
- Effect Size Calculation Procedures in Exercise Intervention Meta-Analyses: A Scoping Review.Sports medicine (Auckland, N.Z.) · 2026Review
- A systematic review and meta-analysis of psychological and behavioural responses in human-agent vs. human-human interactions.Communications psychology · 2026Article
- A Verification Report of Three Meta-Analyses in Transcranial Direct-Current Stimulation-Motor Learning Research.The European journal of neuroscience · 2026Review
- Transparency in Cognitive Training Meta-analyses: A Meta-review.Neuropsychology review · 2025Review
- Reliability of meta-analyses in ecology and evolution: (mostly) good news from a case study on sexual signals.Proceedings. Biological sciences · 2025Article
- Reliability and reproducibility of systematic reviews informing the 2020-2025 Dietary Guidelines for Americans: a pilot study.The American journal of clinical nutrition · 2025Article
- Transparency, quality, and statistical consistency of meta-analytic systematic reviews in clinical child and adolescent psychology (2022-2024): study protocol for a meta-review.Frontiers in psychology · 2025Article
- Reproducibility of meta-analytic results in systematic reviews of interventions: meta-research study.BMJ medicine · 2025Article
- Detecting jingle and jangle fallacies by identifying consistencies and variabilities in study specifications - a call for research.Frontiers in psychology · 2024Article
- Replicability of simulation studies for the investigation of statistical methods: the RepliSims project.Royal Society open science · 2024Article
- Evidence for Infant-directed Speech Preference Is Consistent Across Large-scale, Multi-site Replication and Meta-analysis.Open mind : discoveries in cognitive science · 2024Article
- Research Into Evidence-Based Psychological Interventions Needs a Stronger Focus on Replicability.Clinical psychology in Europe · 2023Article
- Inaccuracy in the Scientific Record and Open Postpublication Critique.Perspectives on psychological science : a journal of the Association for Psychological Science · 2023Article
- Systematic reviewers' perspectives on sharing review data, analytic code, and other materials: A survey.Cochrane evidence synthesis and methods · 2023Article
- Next-Generation Meta-analysis for Next-Generation Questions: Introducing the Prevention Science Special Issue on Modern Meta-analytic Methods.Prevention science : the official journal of the Society for Prevention Research · 2022Article
- Transparency, replicability, and discovery in cognitive aging research: A computational modeling approach.Psychology and aging · 2022Article
- A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020).Behavior research methods · 2022Review
- A Meta-Analysis on Mobile-Assisted Language Learning Applications: Benefits and Risks.Psychologica Belgica · 2022Article
- Future Objectivity Requires Perspective and Forward Combinatorial Meta-Analyses.Frontiers in psychology · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
To determine the reproducibility of psychological meta-analyses, we investigated whether we could reproduce 500 primary study effect sizes drawn from 33 published meta-analyses based on the information given in the meta-analyses, and whether recomputations of primary study effect sizes altered the overall results of the meta-analysis. Results showed that almost half (k = 224) of all sampled primary effect sizes could not be reproduced based on the reported information in the meta-analysis, mostly because of incomplete or missing information on how effect sizes from primary studies were selected and computed. Overall, this led to small discrepancies in the computation of mean effect sizes, confidence intervals and heterogeneity estimates in 13 out of 33 meta-analyses. We provide recommendations to improve transparency in the reporting of the entire meta-analytic process, including the use of preregistration, data and workflow sharing, and explicit coding practices.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.