ArticleBMJ open2020
Outcome reporting bias in Cochrane systematic reviews: a cross-sectional analysis.
Article in BMJ open, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
8 citing papers in PubMed, 1 synthesis or guideline pooled it.
- PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews.BMJ (Clinical research ed.) · 2021Guideline
- Assessment of Selective Reporting Biases in Studies Included in Campbell Systematic Reviews: A Systematic Review.Campbell systematic reviews · 2026Review
- Teaching common cognitive biases and associated errors in veterinary neurologic diagnosis using clinical vignettes: a narrative review.Frontiers in veterinary science · 2026Review
- The threat of serious outcome reporting bias in randomized controlled trials on acute ischemic stroke to evidence synthesis: a meta-epidemiological study.Cardiovascular diagnosis and therapy · 2025Review
- Methodological and Systematic Errors in Systematic Reviews in Health Domain: A Systematic Review.Medical journal of the Islamic Republic of Iran · 2025Review
- PROTOCOL: Assessment of outcome reporting bias in studies included in Campbell systematic reviews.Campbell systematic reviews · 2023Article
- Is augmentation the best solution in partial anterior cruciate ligament tears? A literature systematic review and meta-analysis.Journal of orthopaedics · 2023Review
- Centralised Full Access to Clinical Study Data Can Support Unbiased Guideline Development, Continuing Medical Education, and Patient Information.Journal of European CME · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundDiscrepancies in outcome reporting (DOR) between protocol and published studies include inclusions of new outcomes, omission of prespecified outcomes, upgrade and downgrade of secondary and primary outcomes, and changes in definitions of prespecified outcomes. DOR can result in outcome reporting bias (ORB) when changes in outcomes occur after knowledge of results. This has potential to overestimate treatment effects and underestimate harms. This can also occur at the level of systematic reviews when changes in outcomes occur after knowledge of results of included studies. The prevalence of DOR and ORB in systematic reviews is unknown in systematic reviews published post-2007.
objectiveTo estimate the prevalence of DOR and risk of ORB in all Cochrane reviews between the years 2007 and 2014.
methodsA stratified random sampling approach was applied to collect a representative sample of Cochrane systematic reviews from each Cochrane review group. DOR was assessed by matching outcomes in each systematic review with their respective protocol. When DOR occurred, reviews were further assessed if there was a risk of ORB (unclear, low or high risk). We classified DOR as a high risk for ORB if the discrepancy occurred after knowledge of results in the systematic review.
results150 of 350 (43%) review and protocol pairings contained DOR. When reviews were further scrutinised, 23% (35 of 150) of reviews with DOR contained a high risk of ORB, with changes being made after knowledge of results from individual trials.
conclusionsIn our study, we identified just under a half of Cochrane reviews with at least one DOR. Of these, a fifth were at high risk of ORB. The presence of DOR and ORB in Cochrane reviews is of great concern; however, a solution is relatively simple. Authors are encouraged to be transparent where outcomes change and to describe the legitimacy of changing outcomes in order to prevent suspicion of bias.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.