ArticleEndocrine2024
Assessment of ChatGPT's adherence to ETA-thyroid nodule management guideline over two different time intervals 14 days apart: in binary and multiple-choice queries.
Article in Endocrine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The potential of large language models in the field of infertility: a systematic review.Journal of assisted reproduction and genetics · 2025Pooled it
- Laboratory Medicine Decision Support-Beyond Exam Passing: A Blinded 100-Case Text-Based Benchmark of Diagnostic Accuracy, Management Quality, and Safety for ChatGPT, Gemini, and DeepSeek-LLM Decision Support in Laboratory Medicine.Diagnostics (Basel, Switzerland) · 2026Article
- Article
- Evaluation of ChatGPT-4o in Breast Cancer Screening: Insights from the 5th Edition BI-RADS Atlas and ACR Guidelines.Journal of imaging informatics in medicine · 2026Article
- Benchmarking next-generation large language models in otorhinolaryngology: a prospective, guideline-anchored, blinded evaluation of ChatGPT-5, Gemini-3, Grok-2, Llama-4, and DeepSeek-4 across case scenarios.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Article
- Promise and pitfalls of AI chatbots in complex decision-making for thyroid nodules and papillary thyroid cancer.European thyroid journal · 2026Article
- Assessing the informatics utility of artificial intelligence chatbots for patient and clinician support: the case of adrenal incidentaloma.Endocrine · 2026Article
- Artificial intelligence in endocrine practice: comparing ChatGPT, Gemini, and Claude for adrenal incidentaloma care.Journal of endocrinological investigation · 2026Article
- Evaluation of the effectiveness of ChatGPT in supporting the management of autoimmune hepatitis effectiveness of ChatGPT in autoimmune hepatitis.BMC gastroenterology · 2025Article
- Variability of ChatGPT in Interpreting the Lexicon of ACR-TIRADS, EU-TIRADS, and K-TIRADS.Diagnostics (Basel, Switzerland) · 2025Article
- Applications of Natural Language Processing in Otolaryngology: A Scoping Review.The Laryngoscope · 2025Article
- The impact of prompting on ChatGPT's adherence to status epilepticus treatment guidelines.Scientific reports · 2025Article
- Assessing ChatGPT-v4 for Guideline-Concordant Inflammatory Bowel Disease: Accuracy, Completeness, and Temporal Drift.Journal of clinical medicine · 2025Article
- Assessment of ChatGPT's adherence to EULAR diagnostic criteria and therapeutic protocols for rheumatoid arthritis at two distinct time points, 14 days apart, utilizing binary and multiple-choice inquiries.Clinical rheumatology · 2025Article
- Evaluation of the impact Of ChatGPT support on acromegaly management and patient education.Endocrine · 2025Article
- Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review.BMC medical informatics and decision making · 2024Review
- Assessment of ChatGPT's Compliance with ESC-Acute Coronary Syndrome Management Guidelines at 30-Day Intervals.Life (Basel, Switzerland) · 2024Article
- ChatGPT's Efficacy in Queries Regarding Polycystic Ovary Syndrome and Treatment Strategies for Women Experiencing Infertility.Diagnostics (Basel, Switzerland) · 2024Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveArtificial intelligence (AI) has significant potential in healthcare, particularly in providing decision-support in specialized domains like thyroid nodule management. This study assesses the effectiveness of ChatGPT-v4, an advanced AI model, in aligning with the European Thyroid Association (ETA) - 2023 guidelines.
methodsThe study utilized a structured questionnaire comprising 100 questions, divided into true/false and multiple-choice formats, reflecting real-world clinical scenarios in thyroid nodule management. These questions encompassed diagnostic criteria, treatment options, follow-up protocols, and patient counseling. ChatGPT response was evaluated for accuracy, consistency, and comprehensiveness using a six-point Likert scale. The assessment occurred initially and was repeated after 14 days.
resultsIn the binary queries, the AI model showed an ability to correct some initially incorrect responses. However, there was a noticeable regression in certain responses. 8 of the 11 previously non-compliant responses remained unchanged, while 3 non-compliant responses were rectified. Conversely, 6 initially compliant answers transitioned to non-compliance after 14 days. In multiple-choice queries, the AI's performance was more consistent. A majority of the responses, 43 (86% of the total), were initially correct and maintained their correctness upon re-assessment. However, 4 responses that were initially incorrect remained unchanged, and 3 correct responses shifted to non-compliance over time.
conclusionChatGPT exhibited improving potential as a clinical support tool in thyroid nodule management altgouh it showed varied performance for binary and multiple-choice questions. CLINICAL
trial registrationN/A.
Indexed as
Identifiers
38489133What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.