ArticleBMC anesthesiology2026
A comparative evaluation of the quality of responses provided by different large language model chatbots to frequently asked questions regarding nerve blocks.
Article in BMC anesthesiology, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 3 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
3 citing papers in PubMed.
- Large Language Models in Perioperative Antithrombotic Management: A Blinded Scenario-Based Expert Evaluation.Diagnostics (Basel, Switzerland) · 2026Article
- Comparison of artificial intelligence-based chatbots and expert periodontists in responding to patient questions: a multi-dimensional analysis.BMC oral health · 2026Article
- Large Language Model-Assisted Point-in-Time Interpretation of Advanced Hemodynamics in Liver Transplant Recipients: A Pilot Evaluation of Content Quality and Safety.Journal of clinical medicine · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
STUDY
objectiveLarge language models (LLMs) are used in all areas of life and have become one of the information sources for those seeking healthcare. Although ChatGPT is the most well-known, Claude, CoPilot, and GEMINI are also among the other LLMs. Some of these models have been studied in terms of their response quality metrics to frequently asked questions (FAQs) about broad content areas like anesthesia and to specific FAQs related to obstetric analgesia. However, no studies have yet been conducted on questions related to nerve blocks. In this study, we evaluated the quality of the answers given by the four LLMs to frequently asked questions related to 'nerve block'.
designProspective, Delphi study, Survey.
interventionTen FAQs were identified and presented to four LLMs. A Delphi study was conducted to develop an assessment tool. A survey study was then conducted using the developed tool, in which the evaluators, selected through a thorough process, evaluated the LLM responses. MEASUREMENTS: The quality of LLM responses was assessed by raters using the ARQuAT (Assessing Response Quality in AI Texts) tool, determined through Delphi rounds. Evaluation criteria included content criteria such as accuracy, comprehensiveness, security, timeliness, and relevance, as well as communication criteria such as understandability, empathy, ethical considerations, readability, and neutrality. MAIN
resultsChatGPT and Claude demonstrated superior performance in ARQuAT-Overall scores compared to GEMINI and CoPilot (p < 0.001). ChatGPT and Claude achieved satisfaction rates above 80% in both content and communication quality metrics, significantly outperforming GEMINI (p < 0.001 for both comparisons), while CoPilot showed intermediate performance.
conclusionResponses to FAQs related to nerve blocks were well and acceptably addressed by ChatGPT, Claude, and, to a lesser extent, CoPilot. GEMINI performed poorly compared to the others, exhibiting subpar performance on several questions, particularly in terms of safety and relevance.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.