Raising the bar: Can an Online Module With a Validated Tool Improve Clerkship Evaluations?

Introduction: Narrative evaluations are crucial components of medical education, offering valuable information about students' performance. However, the quality of narratives is often variable. Faculty development interventions to improve narrative evaluations can be time-intensive, and clerkship directors cannot mandate faculty participation. To address this challenge, this study implemented a time-efficient faculty development intervention that used the validated Narrative Evaluation Quality Instrument (NEQI) as a feedback tool to improve the quality of narrative evaluations. Methods: Eighty-nine faculty from the Ambulatory Clerkship at UT Southwestern and the Pediatrics Clerkship at McGovern Medical School were randomized to either a control group, which received no feedback on their narrative evaluations, or an intervention group, which received an educational module on enhancing narrative evaluations using the NEQI, along with individualized NEQI data via email. Pre and post NEQI scores were compared between both groups. Results: While no significant differences in pre and post NEQI scores were found between the two groups and subgroup analysis of faculty who completed the educational module also showed no statistically significant change, the module demonstrated feasibility across the two clerkships. Results revealed a positive trend toward greater specificity in narrative comments. Discussion: The lack of measurable change in NEQI scores may be attributed to the timing of the educational module or limited opportunities for direct student observation. Future research should explore the use of artificial intelligence-assisted scoring and standardized templates to prompt more detailed and actionable narrative evaluations. Clerkship directors need support for faculty education and development of novel interventions to improve the quality of narrative evaluations. Our study demonstrated that simply emailing feedback and an invitation for an asynchronous learning module did not result in measurable improvement in narrative quality.