Evidence map›Paper›PMID 32847800›Full record

ReviewBMJ (Clinical research ed.)2020

Potential impact of missing outcome data on treatment effects in systematic reviews: imputation study.

Lara A Kahale, Assem M Khamis, Batoul Diab, Yaping Chang, Luciane Cruz Lopes, Arnav Agarwal, Ling Li, Reem A Mustafa, Serge Koujanian, Reem Waziry and 7 more

Erratum issuedAbstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in BMJ (Clinical research ed.), 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 16 papers, 5 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
16citing papers in PubMed, 5 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

16 citing papers in PubMed, 5 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Trial
  7. Trial
  8. Article
  9. Review
  10. Review
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

17 authors.

Lara A KahaleDepartment of Internal Medicine, American University of Beirut Medical Center, PO Box 11-0236, Riad-El-Solh Beirut 1107 2020, Beirut, Lebanon.
Assem M KhamisWolfson Palliative Care Research Centre, Hull York Medical School, University of Hull, Hull, UK.
Batoul DiabDepartment of Internal Medicine, American University of Beirut Medical Center, PO Box 11-0236, Riad-El-Solh Beirut 1107 2020, Beirut, Lebanon.
Yaping ChangDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
Luciane Cruz LopesPharmaceutical Sciences Post Graduate Course, University of Sorocaba, UNISO, Sorocaba, Sao Paulo, Brazil.
Arnav AgarwalDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
Ling LiChinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu, China.
Reem A MustafaDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
Serge KoujanianDepartment of Evaluative Clinical Sciences, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
Reem WaziryDepartment of Epidemiology, Harvard TH Chan School of Public Health, Boston, MA, USA.
Jason W BusseDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
Abeer DakikDepartment of Internal Medicine, American University of Beirut Medical Center, PO Box 11-0236, Riad-El-Solh Beirut 1107 2020, Beirut, Lebanon.
Holger J SchünemannDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
Lotty HooftCochrane Netherlands and Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands.
Rob Jpm ScholtenCochrane Netherlands and Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands.
Gordon H GuyattDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
Elie A AklDepartment of Internal Medicine, American University of Beirut Medical Center, PO Box 11-0236, Riad-El-Solh Beirut 1107 2020, Beirut, Lebanon ea32@aub.edu.lb.ORCID 0000-0002-3444-8618

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectiveTo assess the risk of bias associated with missing outcome data in systematic reviews.

designImputation study.

settingSystematic reviews. POPULATION: 100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome.

main outcome measuresMedian percentage change in the relative effect estimate when applying each of the following assumption (four commonly discussed but implausible assumptions (best case scenario, none had the event, all had the event, and worst case scenario) and four plausible assumptions for missing data based on the informative missingness odds ratio (IMOR) approach (IMOR 1.5 (least stringent), IMOR 2, IMOR 3, IMOR 5 (most stringent)); percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. Sensitivity analyses based on the eight different methods of handling missing data were conducted.

results100 systematic reviews with 653 randomised controlled trials were included. When applying the implausible but commonly discussed assumptions, the median change in the relative effect estimate varied from 0% to 30.4%. The percentage of meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario), and 26% changed direction with the worst case scenario. When applying the plausible assumptions, the median percentage change in relative effect estimate varied from 1.4% to 7.0%. The percentage of meta-analyses crossing the threshold of the null effect varied from 6% (IMOR 1.5) to 22% (IMOR 5) of meta-analyses, and 2% changed direction with the most stringent (IMOR 5).

conclusionEven when applying plausible assumptions to the outcomes of participants with definite missing data, the average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. Systematic review authors should present the potential impact of missing outcome data on their effect estimates and use this to inform their overall GRADE (grading of recommendations assessment, development, and evaluation) ratings of risk of bias and their interpretation of the results.

Indexed as

Meta-Analysis as TopicSystematic Reviews as TopicBiasData Interpretation, StatisticalHumansRandomized Controlled Trials as TopicResearch Design

Identifiers

PMID32847800
PMCPMC7448113

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.