ArticleBrain & spine2024
Can AI pass the written European Board Examination in Neurological Surgery? - Ethical and practical issues.
Article in Brain & spine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it, 26 citations in OpenAlex.
- Large language models in neurosurgery: a systematic review and meta-analysis.Acta neurochirurgica · 2024Pooled it
- Publicly Accessible Large Language Model Responses to Frequently Asked Questions About Spondylodiscitis: Preliminary Expert Evaluation.Journal of medical Internet research · 2026Article
- Ethical universality in medicine: A critical rational bioethics beyond cultural relativism.Brain & spine · 2026Article
- Ethical issues in neurosurgery - A scoping review from the EANS Ethico-legal committee.Brain & spine · 2026Review
- Could ChatGPT Be a Tool Capable of Providing Qualified, Empathetic, and Assertive Answers to Patients After Bariatric Surgery? A Comparative Analysis of Its Versions.Obesity surgery · 2025Article
- Can we trust academic AI detective? Accuracy and limitations of AI-output detectors.Acta neurochirurgica · 2025Article
- Employing large language models safely and effectively as a practicing neurosurgeon.Acta neurochirurgica · 2025Review
- Assessing the performance of ChatGPT-4o on the Turkish Orthopedics and Traumatology Board Examination.Joint diseases and related surgery · 2025Article
- Assessing ChatGPT's summarization ofAbdominal radiology (New York) · 2025Article
- Reliability, Accuracy, and Comprehensibility of AI-Based Responses to Common Patient Questions Regarding Spinal Cord Stimulation.Journal of clinical medicine · 2025Article
- Is your curriculum GenAI-proof? A method for GenAI impact assessment and a case study.MedEdPublish (2016) · 2025Article
- Exploring perspectives and boundaries in neurosurgical career pathways for generation Z in German-speaking countries.Brain & spine · 2025Article
- ChatGPT's Performance in Spinal Metastasis Cases-Can We Discuss Our Complex Cases with ChatGPT?Journal of clinical medicine · 2024Article
- Performance of 5 Prominent Large Language Models in Surgical Knowledge Evaluation: A Comparative Analysis.Mayo Clinic proceedings. Digital health · 2024Article
- Assessment Study of ChatGPT-3.5's Performance on the Final Polish Medical Examination: Accuracy in Answering 980 Questions.Healthcare (Basel, Switzerland) · 2024Article
- GPT-4o’s competency in answering the simulated written European Board of Interventional Radiology exam compared to a medical student and experts in Germany and its ability to generate exam items on interventional radiology: a descriptive study.Journal of educational evaluation for health professions · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors at 5 institutions in 4 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Introduction: Artificial intelligence (AI) based large language models (LLM) contain enormous potential in education and training. Recent publications demonstrated that they are able to outperform participants in written medical exams. Research question: We aimed to explore the accuracy of AI in the written part of the EANS board exam. Material and methods: Eighty-six representative single best answer (SBA) questions, included at least ten times in prior EANS board exams, were selected by the current EANS board exam committee. The questions' content was classified as 75 text-based (TB) and 11 image-based (IB) and their structure as 50 interpretation-weighted, 30 theory-based and 6 true-or-false. Questions were tested with Chat GPT 3.5, Bing and Bard. The AI and participant results were statistically analyzed through ANOVA tests with Stata SE 15 (StataCorp, College Station, TX). P-values of <0.05 were considered as statistically significant. Results: The Bard LLM achieved the highest accuracy with 62% correct questions overall and 69% excluding IB, outperforming human exam participants 59% (p = 0.67) and 59% (p = 0.42), respectively. All LLMs scored highest in theory-based questions, excluding IB questions (Chat-GPT: 79%; Bing: 83%; Bard: 86%) and significantly better than the human exam participants (60%; p = 0.03). AI could not answer any IB question correctly. Discussion and conclusion: AI passed the written EANS board exam based on representative SBA questions and achieved results close to or even better than the human exam participants. Our results raise several ethical and practical implications, which may impact the current concept for the written EANS board exam.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.