ArticleSystematic reviews2018
Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and Trial Sequential Analysis.
Article in Systematic reviews, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 86 papers, 48 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
86 citing papers in PubMed, 48 syntheses or guidelines pooled it.
- Effects of blood flow restriction training on cardiovascular autonomic function in middle-aged and older adults: a meta-analysis.Scientific reports · 2026Pooled it
- Cerebral near-infrared spectroscopy monitoring for prevention of death or neurodevelopmental disability in very preterm infants.The Cochrane database of systematic reviews · 2026Pooled it
- Corticosteroids for treatment of leptospirosis.The Cochrane database of systematic reviews · 2025Pooled it
- Silymarin for adults with metabolic dysfunction-associated steatotic liver disease.The Cochrane database of systematic reviews · 2025Pooled it
- Effect of virtual reality-based training on postural balance in children and adolescents with neurodevelopmental disorders: a systematic review and meta-analysis.Frontiers in public health · 2025Pooled it
- A comparative analysis of heme vs non-heme iron administration: a systematic review and meta-analysis of randomized controlled trials.European journal of nutrition · 2024Pooled it
- Vitamin E for people with non-alcoholic fatty liver disease.The Cochrane database of systematic reviews · 2024Pooled it
- Tamoxifen for adults with hepatocellular carcinoma.The Cochrane database of systematic reviews · 2024Pooled it
- Transarterial (chemo)embolisation versus systemic chemotherapy for colorectal cancer liver metastases.The Cochrane database of systematic reviews · 2024Pooled it
- The Omission of Anthracycline Chemotherapy in Women with Early HER2-Negative Breast Cancer-A Systematic Review and Meta-Analysis.Current oncology (Toronto, Ont.) · 2024Pooled it
- Heterologous versus homologous COVID-19 booster vaccinations for adults: systematic review with meta-analysis and trial sequential analysis of randomised clinical trials.BMC medicine · 2024Pooled it
- Antibiotic prophylaxis for leptospirosis.The Cochrane database of systematic reviews · 2024Pooled it
- Antibiotics for treatment of leptospirosis.The Cochrane database of systematic reviews · 2024Pooled it
- Deep neuromuscular blockade in adults undergoing an abdominal laparoscopic procedure.The Cochrane database of systematic reviews · 2024Pooled it
- Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review.Frontiers in psychiatry · 2024Pooled it
- Self-assembling peptides for managing white spot lesions: a systematic review and meta-analysis.European archives of paediatric dentistry : official journal of the European Academy of Paediatric Dentistry · 2023Pooled it
- Granulocyte colony-stimulating factor with or without stem or progenitor cell or growth factors infusion for people with compensated or decompensated advanced chronic liver disease.The Cochrane database of systematic reviews · 2023Pooled it
- Exercise interventions for adults after liver transplantation.The Cochrane database of systematic reviews · 2023Pooled it
- Secondary caries and marginal adaptation of ion-releasing versus resin composite restorations: a systematic review and meta-analysis of randomized clinical trials.Scientific reports · 2022Pooled it
- Prognostic value of novel serum biomarkers, including C-reactive protein to albumin ratio and fibrinogen to albumin ratio, in COVID-19 disease: A meta-analysis.Reviews in medical virology · 2022Pooled it
26 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
backgroundThe evaluation of imprecision is a key dimension of the grading of the confidence in the estimate. Grading of Recommendations Assessment, Development and Evaluation (GRADE) gives recommendations on how to downgrade evidence for imprecision, but authors vary in their use. Trial Sequential Analysis (TSA) has been advocated for a more reliable assessment of imprecision. We aimed to evaluate reporting of and adherence to GRADE and to compare the assessment of imprecision of intervention effects assessed by GRADE and TSA in Cochrane systematic reviews.
methodsIn this cross-sectional study, we included 100 Cochrane reviews irrespective of type of intervention with a key dichotomous outcome meta-analyzed and assessed by GRADE. The methods and results sections of each review were assessed for adequacy of imprecision evaluation. We re-analyzed imprecision following the GRADE Handbook and the TSA Manual.
resultsOverall, only 13.0% of reviews stated the criteria they applied to assess imprecision. The most common dimensions were the 95% width of the confidence intervals and the optimal information size. Review authors downgraded 48.0% of key outcomes due to imprecision. When imprecision was re-analyzed following the GRADE Handbook, 64% of outcomes were downgraded. Agreement between review authors' assessment and assessment by the authors of this study was moderate (kappa 0.43, 95% confidence interval [CI] 0.23 to 0.58). TSA downgraded 69.0% outcomes due to imprecision. Agreement between review authors' GRADE assessment and TSA, irrespective of downgrading levels, was moderate (kappa 0.43, 95% CI 0.21 to 0.57). Agreement between our GRADE assessment following the Handbook and TSA was substantial (kappa 0.66, 95% CI 0.49 to 0.79).
conclusionsIn a sample of Cochrane reviews, methods for assessing imprecision were rarely reported. GRADE according to Handbook guidelines and TSA led to more severe judgment of imprecision rather than GRADE adopted by reviews' authors. Cochrane initiatives to improve adherence to GRADE Handbook are warranted. TSA may transparently assist in such development.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.