Effects of Test Length and Scoring Methods on Pass or Fail Classification Accuracy in Single-Choice Examinations: Simulation Study.

Background: Multiple-choice examinations are widely used in dental and medical education. To reduce the effects of random guessing, several scoring methods incorporating penalty scores have been proposed. However, it remains unclear whether such scoring methods improve the accuracy of pass or fail decisions when pass marks are appropriately calibrated. Objective: This study aimed to examine pass or fail classification accuracy under 3 scoring methods (dichotomous scoring, formula scoring, and right-minus-wrong scoring) in summative examinations using single-choice Type A items with 5 answer options and to evaluate the effects of test length and the distribution of the model parameter Methods: Monte Carlo simulations and analytical derivations were used to evaluate examinations with varying test lengths of up to 300 items. Pass marks were calibrated to the same model-defined cutoff for all scoring methods. Classification accuracy, sensitivity, specificity, and misclassification rates were analyzed across different distributions of Results: With every item answered and pass marks calibrated to the same model-defined cutoff, the 3 scoring methods produced identical pass or fail classifications, as expected from their linear relationship. Classification accuracy increased with test length but was strongly influenced by the distribution of Conclusions: Under the modeled conditions of complete responding, calibrated pass marks, and unchanged response behavior, the investigated scoring methods yielded identical pass or fail classifications. Classification accuracy increased with test length and depended strongly on the distribution of