ArticleBMJ open2019
Comparing single-best-answer and very-short-answer questions for the assessment of applied medical knowledge in 20 UK medical schools: Cross-sectional study.
Article in BMJ open, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 35 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
35 citing papers in PubMed.
- Comparison of cognitive workload between very short answer questions and multiple-choice questions: an eye-tracking experiment.Medical education online · 2026Trial
- Comparison of Online Peer-Assisted Learning and Faculty-Led Teaching for Short Answer Questions.The clinical teacher · 2025Trial
- Trial
- Comparative analysis of short answer questions and multiple-choice questions in formative assessment of first-year medical students.BMC medical education · 2026Article
- Medication-related examination questions in clinical courses within undergraduate medical education.European journal of clinical pharmacology · 2026Article
- Harnessing ChatGPT for SBA question writing: a novel approach to medical student learning.BMC medical education · 2026Article
- Article
- Perceptions from a Single Virtual Near-Peer Mock SBA Examination: A Retrospective Pre-Post Study Among Undergraduate Medical Students.Advances in medical education and practice · 2026Article
- A Systematic Review and Meta-Analysis on the Impact of Peer-Made MCQ Question-Bank Usage on Summative Assessments in Medical Education.Journal of advances in medical education & professionalism · 2025Review
- Development of Immediate Self-Feedback Very Short Answer Questions: Implementing Testlet Response Theory in Formative Examinations Across Multiple Occasions.Medical science educator · 2025Article
- Comparison of mistakes on multiple-choice question and fill-in-the-blank examinations: A retrospective analysis.The Journal of chiropractic education · 2024Article
- Article
- Degenerative Cervical Myelopathy Awareness in Primary Care: UK National Cross-Sectional Survey of General Practitioners.JMIR formative research · 2024Article
- An experimental comparison of multiple-choice and short-answer questions on a high-stakes test for medical students.Advances in health sciences education : theory and practice · 2024Article
- Survey of Ventilator Waveform Interpretation Among ICU Professionals.Respiratory care · 2024Article
- Development and validation of immediate self-feedback very short answer questions for medical students: practical implementation of generalizability theory to estimate reliability in formative examination designs.BMC medical education · 2024Article
- Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study.The Lancet. Digital health · 2024Article
- Assessment approaches in undergraduate health professions education: towards the development of feasible assessment approaches for low-resource settings.BMC medical education · 2024Review
- Understanding medical professionalism using express team-based learning; a qualitative case-based study.Medical education online · 2023Article
- The validity of Engagement and Feedback Assessments (EFAs): identifying students at risk of failing.BMC medical education · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectivesThe study aimed to compare candidate performance between traditional best-of-five single-best-answer (SBA) questions and very-short-answer (VSA) questions, in which candidates must generate their own answers of between one and five words. The primary objective was to determine if the mean positive cue rate for SBAs exceeded the null hypothesis guessing rate of 20%.
designThis was a cross-sectional study undertaken in 2018.
setting20 medical schools in the UK.
participants1417 volunteer medical students preparing for their final undergraduate medicine examinations (total eligible population across all UK medical schools approximately 7500).
interventionsStudents completed a 50-question VSA test, followed immediately by the same test in SBA format, using a novel digital exam delivery platform which also facilitated rapid marking of VSAs.
main outcome measuresThe main outcome measure was the mean positive cue rate across SBAs: the percentage of students getting the SBA format of the question correct after getting the VSA format incorrect. Internal consistency, item discrimination and the pass rate using Cohen standard setting for VSAs and SBAs were also evaluated, and a cost analysis in terms of marking the VSA was performed.
resultsThe study was completed by 1417 students. Mean student scores were 21 percentage points higher for SBAs. The mean positive cue rate was 42.7% (95% CI 36.8% to 48.6%), one-sample t-test against ≤20%: t=7.53, p<0.001. Internal consistency was higher for VSAs than SBAs and the median item discrimination equivalent. The estimated marking cost was £2655 ($3500), with 24.5 hours of clinician time required (1.25 s per student per question).
conclusionsSBA questions can give a false impression of students' competence. VSAs appear to have greater authenticity and can provide useful information regarding students' cognitive errors, helping to improve learning as well as assessment. Electronic delivery and marking of VSAs is feasible and cost-effective.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.