Observational studyEuropean journal of neurology2024
Embracing the future-is artificial intelligence already better? A comparative study of artificial intelligence performance in diagnostic accuracy and decision-making.
Observational study in European journal of neurology, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT07493681 (Role of ChatGPT in the Differential Diagnosis of Polyneuropathies and Comparison of Its Performance With That of Peripheral Neuropathy Specialists and Non-specialists), which is not on this map. Cited by 11 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Role of ChatGPT in the Differential Diagnosis of Polyneuropathies and Comparison of Its Performance With That of Peripheral Neuropathy Specialists and Non-specialists
Who cites it
11 citing papers in PubMed.
- ChatGPT in the diagnosis and management of complex polyneuropathies: comparative analysis with neurologists using real-world cases.NPJ digital medicine · 2026Article
- Applications of Chatbots in Improving Patient Care Outcomes:Sultan Qaboos University medical journal · 2026Article
- Medical QA dialogue datasets in RAG systems performance evaluation and ChatGPT optimization.Scientific reports · 2025Article
- Large Language Models in Neurological Practice: Real-World Study.Journal of medical Internet research · 2025Article
- Large Language Models in Neurology Treatment Decision-Making: a Scoping Review.Journal of medical systems · 2025Article
- Evaluation of deepseek, gemini, ChatGPT-4o, and perplexity in responding to salivary gland cancer.BMC oral health · 2025Article
- Machine Learning-Based Explainable Automated Nonlinear Computation Scoring System for Health Score and an Application for Prediction of Perioperative Stroke: Retrospective Study.Journal of medical Internet research · 2025Article
- Generative artificial intelligence in neurology: Opportunities and risks.European journal of neurology · 2024Article
- Embracing the future-is artificial intelligence already better? A comparative study of artificial intelligence performance in diagnostic accuracy and decision-making.European journal of neurology · 2024Observational
- AI's pivotal impact on redefining stakeholder roles and their interactions in medical education and health care.Frontiers in digital health · 2024Article
- Exploring ChatGPT's potential in the clinical stream of neurorehabilitation.Frontiers in artificial intelligence · 2024Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
background and purposeThe integration of artificial intelligence (AI) in healthcare has the potential to revolutionize patient care and clinical decision-making. This study aimed to explore the reliability of large language models in neurology by comparing the performance of an AI chatbot with neurologists in diagnostic accuracy and decision-making.
methodsA cross-sectional observational study was conducted. A pool of clinical cases from the American Academy of Neurology's Question of the Day application was used as the basis for the study. The AI chatbot used was ChatGPT, based on GPT-3.5. The results were then compared to neurology peers who also answered the questions-a mean of 1500 neurologists/neurology residents.
resultsThe study included 188 questions across 22 different categories. The AI chatbot demonstrated a mean success rate of 71.3% in providing correct answers, with varying levels of proficiency across different neurology categories. Compared to neurology peers, the AI chatbot performed at a similar level, with a mean success rate of 69.2% amongst peers. Additionally, the AI chatbot achieved a correct diagnosis in 85.0% of cases and it provided an adequate justification for its correct responses in 96.1%.
conclusionsThe study highlights the potential of AI, particularly large language models, in assisting with clinical reasoning and decision-making in neurology and emphasizes the importance of AI as a complementary tool to human expertise. Future advancements and refinements are needed to enhance the AI chatbot's performance and broaden its application across various medical specialties.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.