Assessment consistency in special care dentistry: an exploratory study of structured oral examinations.

Background: Assessing competencies in Special Care Dentistry (SCD) is challenging because domain experts are scarce and clinical decision-making is complex. Structured oral examinations (SOEs) may offer a standardized and scalable format for competency-based assessment. Objective: This study aimed to examine whether a SOE supports differentiation between predefined performance levels of SCD-related competencies. Methods: Twenty-four dentists without prior SCD experience independently rated two standardized, video-recorded mock performances representing different competence levels (high vs. moderate) based on the same clinical case. Ratings were performed using a five-point global rating scale (1 = excellent, 5 = fail) supported by a structured expectation guide. Descriptive statistics and inter-rater agreement were examined with particular emphasis on rating distributions and differentiation between performance levels (known-groups approach). Results: The high-quality performance was rated in the highest performance range by all examiners (23 ratings of 1, 1 rating of 2), whereas the moderate performance received lower ratings (13 ratings of 3; 11 ratings of 2). Rating distributions showed minimal variability and a clear separation between performance levels (median 1 vs. 3). Mean scores were 1.04 (SD 0.20) and 2.54 (SD 0.51), respectively. Although the intraclass correlation coefficient was low (ICC=-0.038), this most likely reflects the highly restricted variance inherent in the study design and does not negate the observed consistency of the rating patterns. Conclusion: This exploratory study suggests that structured oral examinations may support differentiation between predefined performance levels in SCD under standardized conditions.