Evidence map›Paper›PMID 36946005›Full record

ArticleClinical and molecular hepatology2023

Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma.

Yee Hui Yeo, Jamil S Samaan, Wee Han Ng, Peng-Sheng Ting, Hirsh Trivedi, Aarshi Vipani, Walid Ayoub, Ju Dong Yang, Omer Liran, Brennan Spiegel and 1 more

2 registry-linked trialsOpen access · goldAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Clinical and molecular hepatology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to 2 registered trials, which are not on this map. Cited by 317 papers, 7 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
317citing papers in PubMed, 7 pooled it
123.2field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT06321328 completednot on this mapstarted 2024, after this paper: background citation

Evaluation of the Success of ChatGPT-4 in Predicting Postoperative Intensive Care Needs and Mortality: Prospective Observational Study

Typeobservational_patient_registrySponsorKanuni Sultan Suleyman Training and Research HospitalRan2024 to 2024Enrolled406ConditionsArtificial Intelligence, Intensive Care Unit, PACU Length of Stay
NCT06321445 completednot on this mapstarted 2024, after this paper: background citation

The Success of ChatGPT in Providing American Society of Anesthesiologist (ASA) Scores

Typeobservational_patient_registrySponsorKanuni Sultan Suleyman Training and Research HospitalRan2024 to 2024Enrolled2,851ConditionsArtificial Intelligence, Preoperative EvaluationArmsproviding ASA score
3 · Its place in the literature

Who cites it

317 citing papers in PubMed, 7 syntheses or guidelines pooled it, 606 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Large Language Models and Empathy: Systematic Review.Journal of medical Internet research · 2024
    Pooled it
  7. Pooled it
  8. Multicenter Evaluation of Large Language Models Versus Hepatologists for Prognostic Prediction in Drug-Induced Liver Injury.Liver international : official journal of the International Association for the Study of the Liver · 2026
    Article
  9. Article
  10. An expert-based assessment of the quality, reliability, understandability, and readability of ChatGPT responses on cerebral palsy.Child's nervous system : ChNS : official journal of the International Society for Pediatric Neurosurgery · 2026
    Article
  11. Review
  12. Article
  13. Article
  14. Article
  15. Review
  16. Article
  17. Article
  18. Article
  19. Review
  20. Article

257 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

11 authors at 3 institutions in 2 countries.

Yee Hui YeoKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Jamil S SamaanKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Wee Han NgBristol Medical School, University of Bristol, Bristol, UK.
Peng-Sheng TingSchool of Medicine, Tulane University, New Orleans, LA, USA.
Hirsh TrivediKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Aarshi VipaniKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Walid AyoubKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Ju Dong YangKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Omer LiranDepartment of Psychiatry and Behavioral Sciences, Cedars-Sinai, Los Angeles, CA, USA.
Brennan SpiegelKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Alexander KuoKarsh Division of Gastroenterology and Hepatology, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
Cedars-Sinai Medical Center · USTulane University · USUniversity of Bristol · GB

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

BACKGROUND/

aimsPatients with cirrhosis and hepatocellular carcinoma (HCC) require extensive and personalized care to improve outcomes. ChatGPT (Generative Pre-trained Transformer), a large language model, holds the potential to provide professional yet patient-friendly support. We aimed to examine the accuracy and reproducibility of ChatGPT in answering questions regarding knowledge, management, and emotional support for cirrhosis and HCC.

methodsChatGPT's responses to 164 questions were independently graded by two transplant hepatologists and resolved by a third reviewer. The performance of ChatGPT was also assessed using two published questionnaires and 26 questions formulated from the quality measures of cirrhosis management. Finally, its emotional support capacity was tested.

resultsWe showed that ChatGPT regurgitated extensive knowledge of cirrhosis (79.1% correct) and HCC (74.0% correct), but only small proportions (47.3% in cirrhosis, 41.1% in HCC) were labeled as comprehensive. The performance was better in basic knowledge, lifestyle, and treatment than in the domains of diagnosis and preventive medicine. For the quality measures, the model answered 76.9% of questions correctly but failed to specify decision-making cut-offs and treatment durations. ChatGPT lacked knowledge of regional guidelines variations, such as HCC screening criteria. However, it provided practical and multifaceted advice to patients and caregivers regarding the next steps and adjusting to a new diagnosis.

conclusionWe analyzed the areas of robustness and limitations of ChatGPT's responses on the management of cirrhosis and HCC and relevant emotional support. ChatGPT may have a role as an adjunct informational tool for patients and physicians to improve outcomes.

Indexed as

Carcinoma, HepatocellularLiver NeoplasmsHumansLiver CirrhosisReproducibility of ResultsArtificial intelligenceChronic disease managementHealth communicationPatient education as topicTelemedicine

Identifiers

PMID36946005
PMCPMC10366809
OpenAlexW4353016766

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.