Evidence map›Paper›PMID 25416499›Full record

ArticleBMJ (Clinical research ed.)2014

Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews.

Pooja Saini, Yoon K Loke, Carrol Gamble, Douglas G Altman, Paula R Williamson, Jamie J Kirkham

Open access · hybridAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in BMJ (Clinical research ed.), 2014. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 85 papers, 22 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
85citing papers in PubMed, 22 pooled it
24.2field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

85 citing papers in PubMed, 22 syntheses or guidelines pooled it, 215 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pregabalin add-on for drug-resistant focal epilepsy.The Cochrane database of systematic reviews · 2022
    Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pregabalin add-on for drug-resistant focal epilepsy.The Cochrane database of systematic reviews · 2019
    Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

25 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

6 authors at 3 institutions in 1 country.

Pooja SainiDepartment of Public Health and Policy, University of Liverpool, Liverpool, UK.
Yoon K LokeNorwich Medical School, University of East Anglia, Norwich, UK.
Carrol GambleDepartment of Biostatistics, University of Liverpool, Liverpool, L69 3GA, UK.
Douglas G AltmanCentre for Statistics in Medicine, University of Oxford, Oxford, UK.
Paula R WilliamsonDepartment of Biostatistics, University of Liverpool, Liverpool, L69 3GA, UK.
Jamie J KirkhamDepartment of Biostatistics, University of Liverpool, Liverpool, L69 3GA, UK jjk@liv.ac.uk.
University of Liverpool · GBUniversity of East Anglia · GBUniversity of Oxford · GB

Funding

Cancer Research UK 16895Cancer Research UK C5529Medical Research Council MR/J004855/1
6 · The paper itself

Abstract

objectiveTo determine the extent and nature of selective non-reporting of harm outcomes in clinical studies that were eligible for inclusion in a cohort of systematic reviews.

designCohort study of systematic reviews from two databases.

settingOutcome reporting bias in trials for harm outcomes (ORBIT II) in systematic reviews from the Cochrane Library and a separate cohort of systematic reviews of adverse events.

participants92 systematic reviews of randomised controlled trials and non-randomised studies published in the Cochrane Library between issue 9, 2012 and issue 2, 2013 (Cochrane cohort) and 230 systematic reviews published between 1 January 2007 and 31 December 2011 in other publications, synthesising data on harm outcomes (adverse event cohort).

methodsA 13 point classification system for missing outcome data on harm was developed and applied to the studies.

results86% (79/92) of reviews in the Cochrane cohort did not include full data from the main harm outcome of interest of each review for all of the eligible studies included within that review; 76% (173/230) for the adverse event cohort. Overall, the single primary harm outcome was inadequately reported in 76% (705/931) of the studies included in the 92 reviews from the Cochrane cohort and not reported in 47% (4159/8837) of the 230 reviews in the adverse event cohort. In a sample of primary studies not reporting on the single primary harm outcome in the review, scrutiny of the study publication revealed that outcome reporting bias was suspected in nearly two thirds (63%, 248/393).

conclusionsThe number of reviews suspected of outcome reporting bias as a result of missing or partially reported harm related outcomes from at least one eligible study is high. The declaration of important harms and the quality of the reporting of harm outcomes must be improved in both primary studies and systematic reviews.

Indexed as

Drug-Related Side Effects and Adverse ReactionsPublishingSelection BiasCohort StudiesData CollectionHumansOutcome and Process Assessment, Health CareRandomized Controlled Trials as TopicResearch DesignReview Literature as TopicTreatment Outcome

Identifiers

PMID25416499
PMCPMC4240443
OpenAlexW2102165930

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.