ArticleEuropean archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery2024
Reliability of large language models for advanced head and neck malignancies management: a comparison between ChatGPT 4 and Gemini Advanced.
Article in European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 24 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
24 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Applications of large language models in tumor boards: a systematic review.Oncology reviews · 2026Pooled it
- Clinical decision support using large language models in otolaryngology: a systematic review.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2025Pooled it
- The doctors of the future: the competition of ChatGPT-4, ChatGPT-4 omni, and Gemini 2.0 Flash in andrology.BMC urology · 2026Article
- Assessment of Stage Two Hypertension Treatment Plans Written by Generative AI.Journal of clinical medicine · 2026Article
- Artificial Intelligence chatbots in oral pathology: a scoping review of applications, accuracy, and limitations.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Article
- Accuracy of large language models in head and neck cancers: a comparative analysis of ChatGPT and Gemini in TNM staging and clinical decision support.Frontiers in oncology · 2026Article
- Comparative Evaluation of Popular Gen-AI Chatbots in Generating Patient Education Material on Pulmonary Artery Catheter Insertion.Annals of cardiac anaesthesia · 2026Article
- Artificial Intelligence Models for Dysphonia Patient Education.Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery · 2025Article
- Artificial intelligence in anesthesia: comparison of the utility of ChatGPT v/s google gemini large language models in pre-anesthetic education: content, readability and sentiment analysis.BMC anesthesiology · 2025Observational
- Temporal evolution of large language models (LLMs) in oncology.Journal of translational medicine · 2025Article
- From Prompts to Practice: Evaluating ChatGPT, Gemini, and Grok Against Plastic Surgeons in Local Flap Decision-Making.Diagnostics (Basel, Switzerland) · 2025Article
- Applications of Natural Language Processing in Otolaryngology: A Scoping Review.The Laryngoscope · 2025Article
- Evaluation of Multiple-Choice Tests in Head and Neck Ultrasound Created by Physicians and Large Language Models.Diagnostics (Basel, Switzerland) · 2025Article
- Assessing LLMs on IDSA Practice Guidelines for the Diagnosis and Treatment of Native Vertebral Osteomyelitis: A Comparison Study.Journal of clinical medicine · 2025Article
- The role of ChatGPT-4o in differential diagnosis and management of vertigo-related disorders.Scientific reports · 2025Article
- The Role of Artificial Intelligence (ChatGPT-4o) in Supporting Tumor Board Decisions.Journal of clinical medicine · 2025Article
- Article
- Chat Generative Pre-Trained Transformer (ChatGPT) in Oral and Maxillofacial Surgery: A Narrative Review on Its Research Applications and Limitations.Journal of clinical medicine · 2025Review
- Evaluating the Efficacy of Artificial Intelligence-Driven Chatbots in Addressing Queries on Vernal Conjunctivitis.Cureus · 2025Article
- Clinical and Pathological Staging Discrepancies in Laryngeal Cancer: A Systematic Review.Cancers · 2025Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeThis study evaluates the efficacy of two advanced Large Language Models (LLMs), OpenAI's ChatGPT 4 and Google's Gemini Advanced, in providing treatment recommendations for head and neck oncology cases. The aim is to assess their utility in supporting multidisciplinary oncological evaluations and decision-making processes.
methodsThis comparative analysis examined the responses of ChatGPT 4 and Gemini Advanced to five hypothetical cases of head and neck cancer, each representing a different anatomical subsite. The responses were evaluated against the latest National Comprehensive Cancer Network (NCCN) guidelines by two blinded panels using the total disagreement score (TDS) and the artificial intelligence performance instrument (AIPI). Statistical assessments were performed using the Wilcoxon signed-rank test and the Friedman test.
resultsBoth LLMs produced relevant treatment recommendations with ChatGPT 4 generally outperforming Gemini Advanced regarding adherence to guidelines and comprehensive treatment planning. ChatGPT 4 showed higher AIPI scores (median 3 [2-4]) compared to Gemini Advanced (median 2 [2-3]), indicating better overall performance. Notably, inconsistencies were observed in the management of induction chemotherapy and surgical decisions, such as neck dissection.
conclusionsWhile both LLMs demonstrated the potential to aid in the multidisciplinary management of head and neck oncology, discrepancies in certain critical areas highlight the need for further refinement. The study supports the growing role of AI in enhancing clinical decision-making but also emphasizes the necessity for continuous updates and validation against current clinical standards to integrate AI into healthcare practices fully.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.