Evidence map›Paper›PMID 29432858›Full record

ArticleJournal of clinical epidemiology2019

GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence.

Holger J Schünemann, Carlos Cuello, Elie A Akl, Reem A Mustafa, Jörg J Meerpohl, Kris Thayer, Rebecca L Morgan, Gerald Gartlehner, Regina Kunz, S Vittal Katikireddi and 4 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Journal of clinical epidemiology, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 445 papers, 119 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
445citing papers in PubMed, 119 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

445 citing papers in PubMed, 119 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Timing of keratinized mucosa augmentation and peri-implant outcomes: A systematic review and meta-analysis.Journal of prosthodontics : official journal of the American College of Prosthodontists · 2026
    Pooled it
  16. Pooled it
  17. Pooled it
  18. Transport and destination hospital for patients with suspected multiple and/or severe injuries - a systematic review and clinical practice guideline update.European journal of trauma and emergency surgery : official publication of the European Trauma Society · 2026
    Pooled it
  19. Pooled it
  20. Pooled it

385 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

14 authors.

Holger J SchünemannDepartment of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada. Electronic address: schuneh@mcmaster.ca.
Carlos CuelloDepartment of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada.
Elie A AklDepartment of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; AUB GRADE Center, Clinical Research Institute, American University of Beirut, PO Box 11-0236, Riad El Solh, Beirut, 1107 2020, Lebanon.
Reem A MustafaDepartment of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; Department of Medicine, University of Kansas Medical Center, 3901 Rainbow Blvd, MS3002, Kansas City, KS, 66160, USA.
Jörg J MeerpohlCochrane Germany, Medical Center University of Freiburg, Breisacher Strasse 153, Freiburg, 79110, Germany.
Kris ThayerIntegrated Risk Information System (IRIS) Division, National Center for Environmental Assessment, Environmental Protection Agency, 1200 Pennsylvania Avenue, N.W.Washington, DC 20460, USA.
Rebecca L MorganDepartment of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada.
Gerald GartlehnerDepartment for Evidence-Based Medicine and Clinical Epidemiology, Danube University Krems, Dr Karl Dorrek Straße 30, Krems, 3500, Austria.
Regina KunzBasel Institute of Clinical Epidemiology, University Hospital Basel, Hebelstrasse 10, Basel, 4031, Switzerland.
S Vittal KatikireddiMRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Top Floor, 200 Renfield Street, Glasgow, G2 3QB, Scotland.
Jonathan SternePopulation Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
Julian Pt HigginsPopulation Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
Gordon GuyattDepartment of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada.
GRADE Working Group

Funding

Chief Scientist Office SCAF/15/02Chief Scientist Office SPHSU13Chief Scientist Office SPHSU15Medical Research Council MC_UU_12017/13Medical Research Council MC_UU_12017/15Medical Research Council MR/M025209/1
6 · The paper itself

Abstract

objectiveTo provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADE's certainty rating process. STUDY DESIGN AND

settingThe study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group.

resultsWe describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I.

conclusionsThe use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.

Indexed as

BiasPractice Guidelines as TopicClinical Trials as TopicEvidence-Based MedicineHumansObservational Studies as TopicRisk AssessmentRisk FactorsSystematic Reviews as TopicUncertaintyCertainty of the evidenceGRADENonrandomized studiesQuality of evidenceRisk of biasROBINS

Identifiers

PMID29432858
PMCPMC6692166

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.