Evidence map›Paper›PMID 42488579›Full record

ArticleNeurosurgery practice2026

Automated Generation and Human Evaluation of Neurosurgical Board Examination Self-Assessment Questions.

Anton Alyakin, Jaden Stryker, Daniel Alexander Alber, Jin Vivian Lee, Shrutika Singh, Akshay Save, David Kurland, Cordelia Orillac, Aly A Valliani, Sean Neifert and 9 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Neurosurgery practice, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

19 authors.

Anton AlyakinDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Jaden StrykerDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Daniel Alexander AlberDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Jin Vivian LeeDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Shrutika SinghDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Akshay SaveDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
David KurlandDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Cordelia OrillacDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Aly A VallianiDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Sean NeifertDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Darryl LauDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Ilya LauferDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Peter A RozmanDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Eveline Teresa HidalgoDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Howard RiinaDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Eric C LeuthardtDepartment of Neurosurgery, Washington University in Saint Louis, Saint Louis, Missouri USA.
Douglas KondziolkaDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.
Laura SnyderDepartment of Neurosurgery, Barrow Neurological Institute, Phoenix, Arizona, USA.
Eric Karl OermannDepartment of Neurological Surgery, NYU Langone Health, New York, New York, USA.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

BACKGROUND AND

objectivesMultiple-choice questions are the primary assessment format for neurosurgical board certification. Creating high-quality examination questions requires significant expert time and resources. The goal of this study was to develop an automated system to generate board-style neurosurgical multiple-choice questions using state-of-the-art vision-language models and compare their quality with authentic self-assessment questions.

methodsWe developed an automated pipeline using OpenAI generative pre-trained transformer (GPT)-4o and Anthropic Claude Sonnet-3.5 to generate neurosurgical board-style questions from

resultsSANS questions were more often perceived as human-made than GPT-generated (residents,

conclusionAlthough quality gaps exist between AI-generated and human-created neurosurgical board examination questions, our approach demonstrates the potential of vision-language models to augment assessment development in specialized medical fields, reducing the burden on examination boards and credentialing organizations.

Indexed as

Board certification preparationEducational content automationImage-based testingMedical education assessmentMultiple-choice question generationVision-language models

Identifiers

PMID42488579
PMCPMC13391137

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.