Mobile Learning in Medical Education: Quasi-Experimental Realist Evaluation of Usage, Context, and Examination Performance in a Curricular Setting.

Background: Mobile learning (mLearning) is widely used in medical education. Previous research has focused on technology acceptance and intervention effectiveness, but rarely on their integration. Using realist evaluation, this study investigated the conditions under which mLearning is adopted and associated with learning-related outcomes in an authentic curricular setting. Objective: This study aimed to examine how learner context and engagement patterns shape mLearning use and outcomes, while secondarily contextualizing its association with examination performance. Methods: A quasi-experimental study was conducted among fifth-semester undergraduate medical students at a German medical school across 2 consecutive summer semesters (2023 and 2024). Students were offered a voluntary, app-based mLearning course in microbiology, delivered via the eSquirrel platform. The course comprised interactive tasks, incorporating elements of gamification and spaced-repetition features. Data sources included nonreactive in-app usage logs, baseline academic performance data, demographic information, and postsemester questionnaire responses. Usage profiles were derived using cluster analysis. Context-mechanism-outcome patterns were explored by relating app usage status to academic performance and survey responses. Results: Of the 245 eligible students, 220 (89.8%) participated in the study; 110 (50%) used the app. In 2024, app users (n=64, 58%) initially appeared to outperform nonusers (n=46, 42%) in the oral microbiology examination (mean grade 2.3, SD 1.1 vs 2.8, SD 1.3; t63,0=1.90; 1-sided P=.03). After adjustment, these differences were largely explained by baseline academic performance, with only limited evidence of an independent intervention effect. Cluster analysis of app users identified 3 engagement profiles: continuous low-intensity use (n=60, 54%), increased use before the examination (n=31, 28%), and use at the beginning of the semester (n=19, 17%). Cluster 2 reported the greatest enjoyment, satisfaction, perceived learning gains, and examination performance in microbiology. Conclusions: Nonreactive in-app usage data provided valuable insights into student engagement. The effectiveness of mLearning was not universal. Examination-oriented use, associated with more strategic and self-regulated study behavior, was linked to more favorable learning outcomes. Future research needs to address equity concerns, as higher-performing students tended to benefit most, as well as explore adaptive, context-sensitive approaches to support diverse learners.