ArticleBJA open2024
Comparison of artificial intelligence large language model chatbots in answering frequently asked questions in anaesthesia.
Article in BJA open, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
18 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The applications of ChatGPT and other large language models in anesthesiology and critical care: a systematic review.Canadian journal of anaesthesia = Journal canadien d'anesthesie · 2025Pooled it
- Article
- Evaluation of Five Large Language Models for Parental Education in Pediatric Anesthesia: Reliability and Readability Study.JMIR medical informatics · 2026Article
- Perioperative Risk Stratification with AI-Powered Chatbots: A Systematic Review and Meta-Analysis.Journal of clinical medicine · 2026Review
- Assessing the Quality, Usefulness, and Reliability of Large Language Models (ChatGPT, DeepSeek, and Gemini) in Answering General Questions Regarding Dyslexia and Dyscalculia.The Psychiatric quarterly · 2026Article
- Performance of Large Language Models in Answering Healthcare Delivery Questions: A Quantitative Cross-Sectional Study.Health science reports · 2026Article
- Evaluation of the accuracy, quality, and readability of large language models on local anesthesia and general anesthesia-sedation in dentistry.BMC anesthesiology · 2026Article
- Artificial Intelligence as a Support Tool for Preoperative Patient Education in Anesthesiology: A Comparative Evaluation of Five Large Language Models.Journal of clinical medicine · 2026Article
- Accuracy, quality, and readability analyses of responses from large language models to questions on pediatric dental sedation.BMC oral health · 2026Article
- Do Large Language Models Perform Equally Across Languages? A Comparison of Responses to Frequently Asked Questions in Anesthesiology.Medical science monitor : international medical journal of experimental and clinical research · 2026Article
- Large Language Model-Assisted Point-in-Time Interpretation of Advanced Hemodynamics in Liver Transplant Recipients: A Pilot Evaluation of Content Quality and Safety.Journal of clinical medicine · 2026Article
- A comparative evaluation of the quality of responses provided by different large language model chatbots to frequently asked questions regarding nerve blocks.BMC anesthesiology · 2026Article
- Comparative analysis of six large language models in perioperative decision support for geriatric patients with multimorbidity: a three-dimensional evaluation framework.BMC anesthesiology · 2026Article
- System-based comparison of the knowledge level of popular AI chatbots on human anatomy: a multiple-choice exam analysis of GPT-4.1, DeepSeek, Co-Pilot, and Gemini models.Surgical and radiologic anatomy : SRA · 2025Article
- Harnessing Generative Artificial Intelligence in Pediatric Anesthesia: Enhancing Learning, Patient Care, and Family Communication.Paediatric anaesthesia · 2025Article
- Emerging Technology and the Future of Perioperative Care: Perspectives and Recommendations From the 2023 Stoelting Conference of the Anesthesia Patient Safety Foundation.Anesthesia and analgesia · 2025Review
- A large language model improves clinicians' diagnostic performance in complex critical illness cases.Critical care (London, England) · 2025Article
- Breaking barriers: harnessing artificial intelligence for a stigma-free, efficient HIV prevention assessment among adults in South Africa.Frontiers in digital health · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
14 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: Patients are increasingly using artificial intelligence (AI) chatbots to seek answers to medical queries. Methods: Ten frequently asked questions in anaesthesia were posed to three AI chatbots: ChatGPT4 (OpenAI), Bard (Google), and Bing Chat (Microsoft). Each chatbot's answers were evaluated in a randomised, blinded order by five residency programme directors from 15 medical institutions in the USA. Three medical content quality categories (accuracy, comprehensiveness, safety) and three communication quality categories (understandability, empathy/respect, and ethics) were scored between 1 and 5 (1 representing worst, 5 representing best). Results: ChatGPT4 and Bard outperformed Bing Chat (median [inter-quartile range] scores: 4 [3-4], 4 [3-4], and 3 [2-4], respectively; Conclusions: In answering anaesthesia patient frequently asked questions, the chatbots perform well on communication metrics but are suboptimal for medical content metrics. Overall, ChatGPT4 and Bard were comparable to each other, both outperforming Bing Chat.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.