Evidence map›Paper›PMID 27519923›Full record

ReviewBMJ open2016

False-positive findings in Cochrane meta-analyses with and without application of trial sequential analysis: an empirical review.

Georgina Imberger, Kristian Thorlund, Christian Gluud, Jørn Wetterslev

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in BMJ open, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 136 papers, 62 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
136citing papers in PubMed, 62 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

136 citing papers in PubMed, 62 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Evaluation ofFrontiers in medicine · 2026
    Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Relationship betweenMedicina (Kaunas, Lithuania) · 2024
    Pooled it

76 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors.

Georgina ImbergerCopenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen University Hospital, Copenhagen, Denmark Department of Anaesthesia & Perioperative Medicine, Monash University, Melbourne, Victoria, Australia.
Kristian ThorlundCopenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen University Hospital, Copenhagen, Denmark Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
Christian GluudCopenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen University Hospital, Copenhagen, Denmark.
Jørn WetterslevCopenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen University Hospital, Copenhagen, Denmark.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectiveMany published meta-analyses are underpowered. We explored the role of trial sequential analysis (TSA) in assessing the reliability of conclusions in underpowered meta-analyses.

methodsWe screened The Cochrane Database of Systematic Reviews and selected 100 meta-analyses with a binary outcome, a negative result and sufficient power. We defined a negative result as one where the 95% CI for the effect included 1.00, a positive result as one where the 95% CI did not include 1.00, and sufficient power as the required information size for 80% power, 5% type 1 error, relative risk reduction of 10% or number needed to treat of 100, and control event proportion and heterogeneity taken from the included studies. We re-conducted the meta-analyses, using conventional cumulative techniques, to measure how many false positives would have occurred if these meta-analyses had been updated after each new trial. For each false positive, we performed TSA, using three different approaches.

resultsWe screened 4736 systematic reviews to find 100 meta-analyses that fulfilled our inclusion criteria. Using conventional cumulative meta-analysis, false positives were present in seven of the meta-analyses (7%, 95% CI 3% to 14%), occurring more than once in three. The total number of false positives was 14 and TSA prevented 13 of these (93%, 95% CI 68% to 98%). In a post hoc analysis, we found that Cochrane meta-analyses that are negative are 1.67 times more likely to be updated (95% CI 0.92 to 2.68) than those that are positive.

conclusionsWe found false positives in 7% (95% CI 3% to 14%) of the included meta-analyses. Owing to limitations of external validity and to the decreased likelihood of updating positive meta-analyses, the true proportion of false positives in meta-analysis is probably higher. TSA prevented 93% of the false positives (95% CI 68% to 98%).

Indexed as

Clinical Trials as TopicMeta-Analysis as TopicStatistics as TopicData Interpretation, StatisticalHumansEPIDEMIOLOGYPUBLIC HEALTHSTATISTICS & RESEARCH METHODS

Identifiers

PMID27519923
PMCPMC4985805

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.