Evidence map›Paper›PMID 37106269›Full record

ArticleObesity surgery2023

Assessing the Accuracy of Responses by the Language Model ChatGPT to Questions Regarding Bariatric Surgery.

Jamil S Samaan, Yee Hui Yeo, Nithya Rajeev, Lauren Hawley, Stuart Abel, Wee Han Ng, Nitin Srinivasan, Justin Park, Miguel Burch, Rabindra Watson and 2 more

Open access · hybridAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Obesity surgery, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 130 papers, 5 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
130citing papers in PubMed, 5 pooled it
10.2field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

130 citing papers in PubMed, 5 syntheses or guidelines pooled it, 266 citations in OpenAlex.

  1. Pooled it
  2. Large language models in obesity: a systematic review.International journal of obesity (2005) · 2026
    Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Trial
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Artificial intelligence, real questions: ChatGPT and cochlear implants.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026
    Article
  17. Review
  18. Article
  19. Article
  20. Article

70 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

12 authors at 3 institutions in 2 countries.

Jamil S SamaanKarsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA, 90048, USA. jamil.samaan@gmail.com.ORCID 0000-0002-6191-2631
Yee Hui YeoKarsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA, 90048, USA.
Nithya RajeevDivision of Upper GI and General Surgery, Department of Surgery, Health Care Consultation Center, Keck School of Medicine of USC, 1510 San Pablo St. #514, Los Angeles, CA, 90033, USA.
Lauren HawleyDivision of Upper GI and General Surgery, Department of Surgery, Health Care Consultation Center, Keck School of Medicine of USC, 1510 San Pablo St. #514, Los Angeles, CA, 90033, USA.
Stuart AbelDivision of Upper GI and General Surgery, Department of Surgery, Health Care Consultation Center, Keck School of Medicine of USC, 1510 San Pablo St. #514, Los Angeles, CA, 90033, USA.
Wee Han NgBristol Medical School, University of Bristol, 5 Tyndall Ave, Bristol, BS8 1UD, UK.
Nitin SrinivasanDivision of Upper GI and General Surgery, Department of Surgery, Health Care Consultation Center, Keck School of Medicine of USC, 1510 San Pablo St. #514, Los Angeles, CA, 90033, USA.
Justin ParkDivision of Upper GI and General Surgery, Department of Surgery, Health Care Consultation Center, Keck School of Medicine of USC, 1510 San Pablo St. #514, Los Angeles, CA, 90033, USA.
Miguel BurchDepartment of Surgery, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA, 90048, USA.
Rabindra WatsonKarsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA, 90048, USA.
Omer LiranDepartment of Psychiatry and Behavioral Sciences, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA, 90048, USA.
Kamran SamakarDivision of Upper GI and General Surgery, Department of Surgery, Health Care Consultation Center, Keck School of Medicine of USC, 1510 San Pablo St. #514, Los Angeles, CA, 90033, USA.
Keck Hospital of USC · USCedars-Sinai Medical Center · USUniversity of Bristol · GB

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeChatGPT is a large language model trained on a large dataset covering a broad range of topics, including the medical literature. We aim to examine its accuracy and reproducibility in answering patient questions regarding bariatric surgery. MATERIALS AND

methodsQuestions were gathered from nationally regarded professional societies and health institutions as well as Facebook support groups. Board-certified bariatric surgeons graded the accuracy and reproducibility of responses. The grading scale included the following: (1) comprehensive, (2) correct but inadequate, (3) some correct and some incorrect, and (4) completely incorrect. Reproducibility was determined by asking the model each question twice and examining difference in grading category between the two responses.

resultsIn total, 151 questions related to bariatric surgery were included. The model provided "comprehensive" responses to 131/151 (86.8%) of questions. When examined by category, the model provided "comprehensive" responses to 93.8% of questions related to "efficacy, eligibility and procedure options"; 93.3% related to "preoperative preparation"; 85.3% related to "recovery, risks, and complications"; 88.2% related to "lifestyle changes"; and 66.7% related to "other". The model provided reproducible answers to 137 (90.7%) of questions.

conclusionThe large language model ChatGPT often provided accurate and reproducible responses to common questions related to bariatric surgery. ChatGPT may serve as a helpful adjunct information resource for patients regarding bariatric surgery in addition to standard of care provided by licensed healthcare professionals. We encourage future studies to examine how to leverage this disruptive technology to improve patient outcomes and quality of life.

Indexed as

Bariatric SurgeryObesity, MorbidHumansLanguageQuality of LifeReproducibility of ResultsArtificial intelligenceBariatric surgeryChatGPTHealth literacyLanguage learning modelsWeight loss

Identifiers

PMID37106269
PMCPMC10234918
OpenAlexW4367175507

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.