Evidence map›Paper›PMID 40745595›Full record

GuidelineBMC medicine2025

Reporting guideline for Chatbot Health Advice studies: the CHART statement.

Bright Huo, Gary Collins, David Chartash, Arun Thirunavukarasu, Annette Flanagin, Alfonso Iorio, Giovanni Cacciamani, Xi Chen, Nan Liu, Piyush Mathur and 37 more

Abstract readGuideline
PubMed Full text at PMC Publisher
In one paragraph

Guideline in BMC medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers.

0numbers the graph read from it
0cells of the map it votes in
9citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

9 citing papers in PubMed.

  1. Article
  2. Bridging the Health Literacy Gap for Patients With Oral Cancer: Readability Enhancement With AI Chatbots.Journal of oral pathology & medicine : official publication of the International Association of Oral Pathologists and the American Academy of Oral Pathology · 2026
    Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

47 authors.

Bright HuoDivision of General Surgery, Department of Surgery, McMaster University, Hamilton, Canada. brighthuo@dal.ca.
Gary CollinsUK EQUATOR Centre, University of Oxford, Oxford, UK.
David ChartashDepartment of Biomedical Informatics and Data Science, Yale University School of Medicine, New Haven, USA.
Arun ThirunavukarasuNuffield Department of Clinical Neurosciences, Medical Sciences Division, University of Oxford, Oxford, UK.
Annette FlanaginJAMA and JAMA Network, American Medical Association, Chicago, USA.
Alfonso IorioDepartment of Health Research Methods, Evidence, and Impact; Department of Medicine, McMaster University, Hamilton, Canada.
Giovanni CacciamaniUSC Institute of Urology and Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
Xi ChenSports Medicine Center, West China Hospital, Sichuan University, Chengdu, China.
Nan LiuDuke-NUS Medical School, National University of Singapore, Singapore, Singapore.
Piyush MathurCleveland Clinic, Case Western Reserve University, Cleveland, USA.
An-Wen ChanDepartment of Medicine, Women's College Research Institute, University of Toronto, Toronto, Canada.
Christine LaineAnnals of Internal Medicine, American College of Physicians, Philadelphia, USA.
Daniela PacellaDepartment of Public Health, University of Naples Federico II, Naples, Italy.
Michael BerkwitsDirector, Office of Science Dissemination, Office of Science, Centers for Disease Control and Prevention, Atlanta, GA, USA.
Stavros A AntoniouDepartment of General Surgery, Papageorgiou General Hospital, Thessaloniki, Greece.
Jennifer C CamaradouBritish Psychological Society, University of Plymouth, Plymouth, UK.
Carolyn CanfieldInnovation Support Unit, Department of Family Practice, University of British Columbia, Vancouver, Canada.
Michael MittelmanPatient SME, Independent Cybersecurity Professional, London, UK.
Timothy FeeneyThe BMJ, London, UK.
Elizabeth LoderThe BMJ, London, UK.
Riaz AghaInternational Journal of Surgery, London, UK.
Ashirbani SahaDepartment of Oncology, McMaster University, Hamilton, Canada.
Julio MayolHospital Clinico San Carlos, Instituto de Investigación Sanitaria San Carlos, Facultad de Medicina Universidad Complutense de Madrid, Madrid, Spain.
Anthony SunjayaThe George Institute for Global Health; Tyree Institute of Health Engineering, UNSW Engineering; School of Population Health, UNSW Medicine and Health, Sydney, Australia.
Hugh HarveyHardian Health, London, UK.
Jeremy Y NgCentre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada.
Tyler McKechnieDivision of General Surgery, Department of Surgery, McMaster University, Hamilton, Canada.
Yung LeeDivision of General Surgery, Department of Surgery, McMaster University, Hamilton, Canada.
Nipun VermaPostgraduate Institute of Medical Education and Research, Chandigarh, India.
Gregor StiglicUniversity of Maribor, Maribor, Slovenia.
Melissa McCraddenAustralian Institute for Machine Learning (AIML), Adelaide, Australia.
Karim RamjiPhelix AI, Toronto, Canada.
Vanessa BoudreauDivision of General Surgery, Department of Surgery, McMaster University, Hamilton, Canada.
Monica OrtenziUniversità Politecnica delle Marche, Clinica di Chirurgia Generale e d'Urgenza, Ancona, Italy.
Joerg MeerpohlInstitute for Evidence in Medicine, Medical Center & Faculty of Medicine, University of Freiburg, Freiburg im Breisgau, Germany.
Per Olav VandvikCochrane Germany, Cochrane Germany Foundation, Freiburg, Germany.
Thomas AgoritsasDepartment of Health Research Methods, Evidence, and Impact; Department of Medicine, McMaster University, Hamilton, Canada.
Diana SamuelThe Lancet Digital Health, London, UK.
Helen FrankishThe Lancet, London, UK.
Michael AndersonNIHR Clinical Lecturer, Health Organisation, Policy, Economics (HOPE), Centre for Primary Care & Health Services Research, The University of Manchester, Manchester, UK.
Xiaomei YaoDepartment of Oncology, McMaster University, Hamilton, Canada.
Stacy LoebNew York University Langone Health, New York City, USA.
Cynthia LokkerDepartment of Health Research Methods, Evidence, and Impact; Department of Medicine, McMaster University, Hamilton, Canada.
Xiaoxuan LiuCollege of Medicine and Health, University of Birmingham, Birmingham, UK.
Eliseo GuallarSchool of Global Public Health, New York University, New York City, USA.
Gordon GuyattDepartment of Health Research Methods, Evidence, and Impact; Department of Medicine, McMaster University, Hamilton, Canada.
CHART Collaborative

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundThe Chatbot Assessment Reporting Tool (CHART) is a reporting guideline developed to provide reporting recommendations for studies evaluating the performance of generative artificial intelligence (AI)-driven chatbots when summarizing clinical evidence and providing health advice, referred to as Chatbot Health Advice (CHA) studies.

methodsCHART was developed in several phases after performing a comprehensive systematic review to identify variation in the conduct, reporting, and methodology in CHA studies. Findings from the review were used to develop a draft checklist that was revised through an international, multidisciplinary modified asynchronous Delphi consensus process of 531 stakeholders, three synchronous panel consensus meetings of 48 stakeholders, and subsequent pilot testing of the checklist.

resultsCHART includes 12 items and 39 subitems to promote transparent and comprehensive reporting of CHA studies. These include Title (subitem 1a), Abstract/Summary (subitem 1b), Background (subitems 2ab), Model Identifiers (subitems 3ab), Model Details (subitems 4abc), Prompt Engineering (subitems 5ab), Query Strategy (subitems 6abcd), Performance Evaluation (subitems 7ab), Sample Size (subitem 8), Data Analysis (subitem 9a), Results (subitems 10abc), Discussion (subitems 11abc), Disclosures (subitem 12a), Funding (subitem 12b), Ethics (subitem 12c), Protocol (subitem 12d), and Data Availability (subitem 12e).

conclusionThe CHART checklist and corresponding methodological diagram were designed to support key stakeholders including clinicians, researchers, editors, peer reviewers, and readers in reporting, understanding, and interpreting the findings of CHA studies.

Indexed as

Artificial IntelligenceResearch DesignChecklistDelphi TechniqueGenerative Artificial IntelligenceHumansGenerative AILLMsReporting standards

Identifiers

PMID40745595
PMCPMC12315282

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.