ArticleBMC medical education2022
Standard setting Very Short Answer Questions (VSAQs) relative to Single Best Answer Questions (SBAQs): does having access to the answers make a difference?
Article in BMC medical education, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 4 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
4 citing papers in PubMed, 1 synthesis or guideline pooled it, 9 citations in OpenAlex.
- Angoff methods in standard setting in health professional education: a systematic review and meta-analysis.BMC medical education · 2025Pooled it
- Perceptions from a Single Virtual Near-Peer Mock SBA Examination: A Retrospective Pre-Post Study Among Undergraduate Medical Students.Advances in medical education and practice · 2026Article
- Development of Immediate Self-Feedback Very Short Answer Questions: Implementing Testlet Response Theory in Formative Examinations Across Multiple Occasions.Medical science educator · 2025Article
- Development and validation of immediate self-feedback very short answer questions for medical students: practical implementation of generalizability theory to estimate reliability in formative examination designs.BMC medical education · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 4 institutions in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundWe investigated whether question format and access to the correct answers affect the pass mark set by standard-setters on written examinations.
methodsTrained educators used the Angoff method to standard set two 50-item tests with identical vignettes, one in a single best answer question (SBAQ) format (with five answer options) and the other in a very short answer question (VSAQ) format (requiring free text responses). Half the participants had access to the correct answers and half did not. The data for each group were analysed to determine if the question format or having access to the answers affected the pass mark set.
resultsA lower pass mark was set for the VSAQ test than the SBAQ test by the standard setters who had access to the answers (median difference of 13.85 percentage points, Z = -2.82, p = 0.002). Comparable pass marks were set for the SBAQ test by standard setters with and without access to the correct answers (60.65% and 60.90% respectively). A lower pass mark was set for the VSAQ test when participants had access to the correct answers (difference in medians -13.75 percentage points, Z = 2.46, p = 0.014).
conclusionsWhen given access to the potential correct answers, standard setters appear to appreciate the increased difficulty of VSAQs compared to SBAQs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.