Evidence map›Paper›PMID 18769481›Full record

SynthesisPloS one2008

Systematic review of the empirical evidence of study publication bias and outcome reporting bias.

Kerry Dwan, Douglas G Altman, Juan A Arnaiz, Jill Bloom, An-Wen Chan, Eugenia Cronin, Evelyne Decullier, Philippa J Easterbrook, Erik Von Elm, Carrol Gamble and 4 more

Abstract readSystematic Review
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in PloS one, 2008. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 507 papers, 50 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
507citing papers in PubMed, 50 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

507 citing papers in PubMed, 50 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Comparative efficacy and safety of the treatment by Omalizumab for chronic idiopathic urticaria in the general population: A systematic review and network meta-analysis.Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging (ISSI) · 2024
    Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. A Systematic Review of Workplace Physical Activity Coaching.Journal of occupational rehabilitation · 2023
    Pooled it
  15. Pooled it
  16. Systematic review of conservation interventions to promote voluntary behavior change.Conservation biology : the journal of the Society for Conservation Biology · 2023
    Pooled it
  17. Pooled it
  18. Pooled it
  19. Area-level racial prejudice and health: A systematic review.Health psychology : official journal of the Division of Health Psychology, American Psychological Association · 2022
    Pooled it
  20. Pooled it

447 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

14 authors.

Kerry DwanCentre for Medical Statistics and Health Evaluation, University of Liverpool, Liverpool, United Kingdom. kerry.dwan@liverpool.ac.uk
Douglas G Altman
Juan A Arnaiz
Jill Bloom
An-Wen Chan
Eugenia Cronin
Evelyne Decullier
Philippa J Easterbrook
Erik Von Elm
Carrol Gamble
Davina Ghersi
John P A Ioannidis
John Simes
Paula R Williamson

Funding

Cancer Research UKMedical Research Council G0200585
6 · The paper itself

Abstract

backgroundThe increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL

findingsWe review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.

conclusionsRecent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Indexed as

Publication BiasRandomized Controlled Trials as TopicClinical Trials as TopicCohort StudiesHumansPatient SelectionPublishingTreatment Outcome

Identifiers

PMID18769481
PMCPMC2518111

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.