ArticleBJA open2024
A comparative study of English and Japanese ChatGPT responses to anaesthesia-related medical questions.
Article in BJA open, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The applications of ChatGPT and other large language models in anesthesiology and critical care: a systematic review.Canadian journal of anaesthesia = Journal canadien d'anesthesie · 2025Pooled it
- Article
- Elucidating the transformative role of large language models in advancing anesthesiology education.The Journal of international medical research · 2026Review
- Quality, completeness, readability, and response time of AI chatbots in the management of deep caries and pulp exposure: a bilingual comparative study.BMC oral health · 2026Article
- Comparing the Accuracy of ChatGPT-4o, DeepSeek-V3, and Gemini 2.5 Flash in Answering Frequently Asked Questions About Systemic Lupus Erythematosus: Quantitative Study.JMIR formative research · 2026Article
- Language-Specific Differences in Large Language Model Diagnostic Reasoning: A Translation-Controlled Clinical Vignette Study.Journal of clinical medicine · 2026Article
- Deciphering AI-chatbots utilization in clinical practice, research and education among Chinese medical students in generative AI era: a cross-sectional study.BMC medical education · 2026Article
- Do Large Language Models Perform Equally Across Languages? A Comparison of Responses to Frequently Asked Questions in Anesthesiology.Medical science monitor : international medical journal of experimental and clinical research · 2026Article
- Large Language Model-Assisted Point-in-Time Interpretation of Advanced Hemodynamics in Liver Transplant Recipients: A Pilot Evaluation of Content Quality and Safety.Journal of clinical medicine · 2026Article
- A comparative evaluation of the quality of responses provided by different large language model chatbots to frequently asked questions regarding nerve blocks.BMC anesthesiology · 2026Article
- Article
- Multidimensional assessment of large language model responses to patient questions on gestational diabetes mellitus.Scientific reports · 2025Article
- Harnessing Generative Artificial Intelligence in Pediatric Anesthesia: Enhancing Learning, Patient Care, and Family Communication.Paediatric anaesthesia · 2025Article
- Performance of Open-Source Large Language Models in Psychiatry: Usability Study Through Comparative Analysis of Non-English Records and English Translations.Journal of medical Internet research · 2025Article
- Performance of DeepSeek-R1 and ChatGPT-4o on the Chinese National Medical Licensing Examination: A Comparative Study.Journal of medical systems · 2025Article
- Enhancing perinatal health patient information through ChatGPT - An accuracy study.PEC innovation · 2025Article
- DeepSeek-R1 outperforms Gemini 2.0 Pro, OpenAI o1, and o3-mini in bilingual complex ophthalmology reasoning.Advances in ophthalmology practice and researchArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: The expansion of artificial intelligence (AI) within large language models (LLMs) has the potential to streamline healthcare delivery. Despite the increased use of LLMs, disparities in their performance particularly in different languages, remain underexplored. This study examines the quality of ChatGPT responses in English and Japanese, specifically to questions related to anaesthesiology. Methods: Anaesthesiologists proficient in both languages were recruited as experts in this study. Ten frequently asked questions in anaesthesia were selected and translated for evaluation. Three non-sequential responses from ChatGPT were assessed for content quality (accuracy, comprehensiveness, and safety) and communication quality (understanding, empathy/tone, and ethics) by expert evaluators. Results: Eight anaesthesiologists evaluated English and Japanese LLM responses. The overall quality for all questions combined was higher in English compared with Japanese responses. Content and communication quality were significantly higher in English compared with Japanese LLMs responses (both Conclusions: English LLM responses to anaesthesia-related frequently asked questions were superior in quality to Japanese responses when assessed by bilingual anaesthesia experts in this report. This study highlights the potential for language-related disparities in healthcare information and the need to improve the quality of AI responses in underrepresented languages. Future studies are needed to explore these disparities in other commonly spoken languages and to compare the performance of different LLMs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.