Evidence map›Paper›PMID 32980429›Full record

ArticleJournal of clinical epidemiology2021

GRADE Guidelines 30: the GRADE approach to assessing the certainty of modeled evidence-An overview in the context of health decision-making.

Jan L Brozek, Carlos Canelo-Aybar, Elie A Akl, James M Bowen, John Bucher, Weihsueh A Chiu, Mark Cronin, Benjamin Djulbegovic, Maicon Falavigna, Gordon H Guyatt and 35 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Journal of clinical epidemiology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 179 papers, 86 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
179citing papers in PubMed, 86 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

179 citing papers in PubMed, 86 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Artificial Intelligence for Working Length Determination in Endodontics: A Systematic Review and Meta-Analysis.Australian endodontic journal : the journal of the Australian Society of Endodontology Inc · 2026
    Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

119 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

45 authors.

Jan L BrozekDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada; McMaster GRADE Centre & Michael DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada.
Carlos Canelo-AybarDepartment of Paediatrics, Obstetrics and Gynaecology, Preventive Medicine, and Public Health. PhD Programme in Methodology of Biomedical Research and Public Health. Universitat Autònoma de Barcelona, Bellaterra, Spain; Iberoamerican Cochrane Center, Biomedical Research Institute (IIB Sant Pau-CIBERESP), Barcelona, Spain.
Elie A AklDepartment of Internal Medicine, American University of Beirut, Beirut, Lebanon.
James M BowenDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Toronto Health Economics and Technology Assessment (THETA) Collaborative, Toronto, Ontario, Canada.
John BucherNational Toxicology Program, National Institute of Environmental Health Sciences, Durham, NC, USA.
Weihsueh A ChiuDepartment of Veterinary Integrative Biosciences, Texas A&M University, College Station, TX, USA.
Mark CroninSchool of Pharmacy and Chemistry, Liverpool John Moores University, Liverpool, UK.
Benjamin DjulbegovicCenter for Evidence-Based Medicine and Health Outcome Research, Morsani College of Medicine, University of South Florida, Tampa, Florida, USA.
Maicon FalavignaInstitute for Education and Research, Hospital Moinhos de Vento, Porto Alegre, Rio Grande do Sul, Brazil.
Gordon H GuyattDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada; McMaster GRADE Centre & Michael DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada.
Ami A GordonICF International, Durham, NC, USA.
Michele Hilton BoonInstitute of Health & Wellbeing, University of Glasgow, Glasgow, UK.
Raymond C W HutubessyDepartment of Immunization, Vaccines and Biologicals, World Health Organization, Geneva, Switzerland.
Manuela A JooreClinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre+, Maastricht, the Netherlands.
Vittal KatikireddiInstitute of Health & Wellbeing, University of Glasgow, Glasgow, UK.
Judy LaKindLaKind Associates, LLC, Catonsville, MD, USA; Department of Epidemiology and Public Health, University of Maryland School of Medicine, Baltimore, MD, USA.
Miranda LangendamDepartment of Clinical Epidemiology, Biostatistics, and Bioinformatics, Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands.
Veena ManjaDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Surgery, University of California Davis, Sacramento, CA, USA; Department of Medicine, Department of Veterans Affairs, Northern California Health Care System, Mather, CA, USA.
Kristen MagnusonICF International, Durham, NC, USA.
Alexander G MathioudakisDivision of Infection, Immunity and Respiratory Medicine, University Hospital of South Manchester, University of Manchester, Manchester, UK.
Joerg MeerpohlInstitute for Evidence in Medicine, Medical Center, University of Freiburg, Freiburg-am-Breisgau, Germany; Cochrane Germany, Freiburg-am-Breisgau, Germany.
Dominik MertzDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
Roman MezencevNational Center for Environmental Assessment, U.S. Environmental Protection Agency, Washington, DC, USA.
Rebecca MorganDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
Gian Paolo MorganoDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; McMaster GRADE Centre & Michael DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada.
Reem MustafaDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
Martin O'FlahertyInstitute of Population Health Sciences, University of Liverpool, Liverpool, UK.
Grace PatlewiczNational Center for Computational Toxicology, U.S. Environmental Protection Agency, Durham, NC, USA.
John J RivaMcMaster GRADE Centre & Michael DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada; Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada.
Margarita PossoIberoamerican Cochrane Center, Biomedical Research Institute (IIB Sant Pau-CIBERESP), Barcelona, Spain.
Andrew RooneyNational Toxicology Program, National Institute of Environmental Health Sciences, Durham, NC, USA.
Paul M SchlosserNational Center for Environmental Assessment, U.S. Environmental Protection Agency, Washington, DC, USA.
Lisa SchwartzDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
Ian ShemiltEPPI-Centre, Institute of Education, University College London, London, UK.
Jean-Eric TarrideDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Programs for Assessment of Technology in Health, McMaster University, Hamilton, Ontario, Canada.
Kristina A ThayerDepartment of Medicine, Department of Veterans Affairs, Northern California Health Care System, Mather, CA, USA.
Katya TsaiounEvidence-Based Toxicology Collaboration, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
Luke ValeHealth Economics Group, Institute of Health and Society, Newcastle University, Newcastle upon Tyne, UK.
John WambaughNational Center for Computational Toxicology, U.S. Environmental Protection Agency, Durham, NC, USA.
Jessica WignallICF International, Durham, NC, USA.
Ashley WilliamsICF International, Durham, NC, USA.
Feng XieDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
Yuan ZhangDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Health Quality Ontario, Toronto, Ontario, Canada.
Holger J SchünemannDepartment of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada; McMaster GRADE Centre & Michael DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada.
GRADE Working Group

Funding

Integrated Health Assessments for the Division of the National Toxicology ProgramZICES103317 · NIEHS · NATIONAL INSTITUTE OF ENVIRONMENTAL HEALTH SCIENCES · PI ROONEY, ANDREW · 2020 to 2021
$9.6M
Intramural NIH HHS Z99 ES999999Medical Research Council MR/K02325X/1World Health Organization 001
6 · The paper itself

Abstract

objectivesThe objective of the study is to present the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) conceptual approach to the assessment of certainty of evidence from modeling studies (i.e., certainty associated with model outputs). STUDY DESIGN AND

settingExpert consultations and an international multidisciplinary workshop informed development of a conceptual approach to assessing the certainty of evidence from models within the context of systematic reviews, health technology assessments, and health care decisions. The discussions also clarified selected concepts and terminology used in the GRADE approach and by the modeling community. Feedback from experts in a broad range of modeling and health care disciplines addressed the content validity of the approach.

resultsWorkshop participants agreed that the domains determining the certainty of evidence previously identified in the GRADE approach (risk of bias, indirectness, inconsistency, imprecision, reporting bias, magnitude of an effect, dose-response relation, and the direction of residual confounding) also apply when assessing the certainty of evidence from models. The assessment depends on the nature of model inputs and the model itself and on whether one is evaluating evidence from a single model or multiple models. We propose a framework for selecting the best available evidence from models: 1) developing de novo, a model specific to the situation of interest, 2) identifying an existing model, the outputs of which provide the highest certainty evidence for the situation of interest, either "off-the-shelf" or after adaptation, and 3) using outputs from multiple models. We also present a summary of preferred terminology to facilitate communication among modeling and health care disciplines.

conclusionThis conceptual GRADE approach provides a framework for using evidence from models in health decision-making and the assessment of certainty of evidence from a model or models. The GRADE Working Group and the modeling community are currently developing the detailed methods and related guidance for assessing specific domains determining the certainty of evidence from models across health care-related disciplines (e.g., therapeutic decision-making, toxicology, environmental health, and health economics).

Indexed as

GRADE ApproachClinical Decision-MakingEvidence-Based MedicineHumansInterdisciplinary CommunicationProfessional CompetencePublication BiasSystematic Reviews as TopicTechnology Assessment, BiomedicalCertainty of evidenceGRADEGuidelinesHealth care Decision makingMathematical modelsModelling studies

Identifiers

PMID32980429
PMCPMC8514123

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.