ArticleBMJ (Clinical research ed.)2011
Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study.
Article in BMJ (Clinical research ed.), 2011. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 22 papers, 12 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
22 citing papers in PubMed, 12 syntheses or guidelines pooled it.
- Effectiveness of Telehealth Care Interventions for Women's Mental Health During and After the COVID-19 Pandemic: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2026Pooled it
- Mineralocorticoid receptor antagonists and mortality in hypertension: a systematic review and meta-analysis.Journal of hypertension · 2026Pooled it
- Psychological Therapy Quantity and Depressive Symptom Reduction in Psychedelic-Assisted Therapy: A Systematic Review and Meta-Analysis.JAMA network open · 2026Pooled it
- Drugs for Procedural Sedation and Analgesia in Children: A Systematic Review and Meta-analysis.Drugs in R&D · 2025Pooled it
- Retinal sensitivity changes in early/intermediate AMD: a systematic review and meta-analysis of visual field testing under mesopic and scotopic lighting.Eye (London, England) · 2024Pooled it
- Efficacy of psilocybin for treating symptoms of depression: systematic review and meta-analysis.BMJ (Clinical research ed.) · 2024Pooled it
- Effects of renin-angiotensin system blockers on outcomes from COVID-19: a systematic review and meta-analysis of randomized controlled trials.European heart journal. Cardiovascular pharmacotherapy · 2024Pooled it
- Antiplatelet therapy and incident cognitive impairment or dementia-a systematic review and meta-analysis of randomised clinical trials.Age and ageing · 2023Pooled it
- Pooled it
- Pooled it
- Predictors of persistent pain after breast cancer surgery: a systematic review and meta-analysis of observational studies.CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne · 2016Pooled it
- Pooled it
- Should we consider microbiota-based interventions as a novel therapeutic strategy for schizophrenia? A systematic review and meta-analysis.Brain, behavior, & immunity - health · 2025Article
- A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020).Behavior research methods · 2022Review
- Article
- Article
- Standards for design and measurement would make clinical research reproducible and usable.Proceedings of the National Academy of Sciences of the United States of America · 2018Review
- Investigation of bias in meta-analyses due to selective inclusion of trial effect estimates: empirical study.BMJ open · 2016Article
- Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions.The Cochrane database of systematic reviews · 2014Article
- An empirical investigation of the potential impact of selective inclusion of results in systematic reviews of interventions: study protocol.Systematic reviews · 2013Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
objectivesTo examine the extent of multiplicity of data in trial reports and to assess the impact of multiplicity on meta-analysis results.
designEmpirical study on a cohort of Cochrane systematic reviews. DATA SOURCES: All Cochrane systematic reviews published from issue 3 in 2006 to issue 2 in 2007 that presented a result as a standardised mean difference (SMD). We retrieved trial reports contributing to the first SMD result in each review, and downloaded review protocols. We used these SMDs to identify a specific outcome for each meta-analysis from its protocol. Review methods Reviews were eligible if SMD results were based on two to ten randomised trials and if protocols described the outcome. We excluded reviews if they only presented results of subgroup analyses. Based on review protocols and index outcomes, two observers independently extracted the data necessary to calculate SMDs from the original trial reports for any intervention group, time point, or outcome measure compatible with the protocol. From the extracted data, we used Monte Carlo simulations to calculate all possible SMDs for every meta-analysis.
resultsWe identified 19 eligible meta-analyses (including 83 trials). Published review protocols often lacked information about which data to choose. Twenty-four (29%) trials reported data for multiple intervention groups, 30 (36%) reported data for multiple time points, and 29 (35%) reported the index outcome measured on multiple scales. In 18 meta-analyses, we found multiplicity of data in at least one trial report; the median difference between the smallest and largest SMD results within a meta-analysis was 0.40 standard deviation units (range 0.04 to 0.91).
conclusionsMultiplicity of data can affect the findings of systematic reviews and meta-analyses. To reduce the risk of bias, reviews and meta-analyses should comply with prespecified protocols that clearly identify time points, intervention groups, and scales of interest.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.