ArticleEuropean archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery2026
Experts V/S AI´s 2.0: Comparative evaluation of AI models and expert consensus in obstructive sleep apnea assessment.
Article in European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 3 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
3 citing papers in PubMed.
- Standardizing AI evaluation in sleep medicine: a reply to Rani et al.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Article
- Comment on "Experts V/S AI´s 2.0: comparative evaluation of ai models and expert consensus in obstructive sleep apnea assessment".European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Article
- Telemedicine in Oral and Maxillofacial Surgery: A Narrative Review of Clinical Applications, Outcomes and Future Directions.Journal of clinical medicine · 2026Review
Corrections and comments
- Commented on by
Authors and funding
16 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeThis study aims to compare the evaluation of obstructive sleep apnea (OSA) by ten super-experts using responses from a 10-question survey answered by 3 different artificial intelligence chatbots, Chat GPT-3.5, Chat GPT-4.0, Gemini, and a panel of 100 otolaryngologists specialized in sleep medicine.
methodsA 10-question survey regarding OSA management was answered by Chat GPT-3.5, Chat GPT-4.0, Gemini, and a panel of 100 otolaryngologists. The responses were assessed by ten super-experts in sleep medicine for their agreement with expert consensus, using a Likert scale. Statistical analyses were performed to evaluate the level of agreement and significance.
resultExpert consensus had the highest mean score (4.5 ± 0.9), significantly outperforming all AI models. ChatGPT-3.5 was the best among AI systems, with a score of 4.1 ± 1.2 (p=0.003), followed by ChatGPT-4 with 3.9 ± 1.4 (p<0.001) and Gemini with 3.6 ± 1.5 (p<0.001). Perfect agreement with expert consensus was achieved in specific scenarios, particularly regarding indications for bariatric surgery and lateral pharyngoplasty. However, there were significant differences in complex clinical scenarios that required integration of multiple factors, particularly in therapeutic management questions where the performance of AI models was significantly below that of expert consensus (p<0.01).
conclusionsAlthough AI models are promising in the management of OSA, especially for well-defined clinical scenarios, they at present serve best as complementary tools rather than replacements for expert clinical judgment. Most surprisingly, ChatGPT-3.5 outperformed its newer versions in many aspects, indicating that model updates with general capabilities may not always lead to better performance in specialized medical domains. These findings emphasize the potential of AI as a supportive resource while emphasizing the continuing need for human expertise in complex clinical decision-making.
Indexed as
Identifiers
41176557What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.