Evaluating the Influence of MCAT Scores on Medical Student Selection and Performance Using a Machine Learning Approach: Retrospective Cohort Study.

Background: The Medical College Admission Test (MCAT) has been central to medical school admissions in North America, though its necessity in holistic selection processes remains debated. The COVID-19 pandemic's suspension of MCAT testing sessions allowed institutions to explore alternative admission criteria. Additionally, global challenges in test administration underscore the vulnerability of systems dependent on a single standardized test. Objective: This study aimed to (1) assess whether including MCAT scores in admissions decisions improves the prediction of medical school performance compared with grade point average (GPA) and interview-based selection and (2) evaluate whether machine learning (ML) can generate viable MCAT score predictions when testing is unavailable. Methods: We conducted a retrospective cohort study of 1898 applicants to the Lebanese American University School of Medicine (2009-2023). Among 349 admitted students with MCAT scores, we compared admission composites including vs excluding the MCAT in relation to academic outcomes (Med 1-4 final grades) and clinical performance (Med 1-2 objective structured clinical examination [OSCE] scores). Students were stratified into tertiles to examine tier-specific effects. To model an MCAT alternative, we trained an ensemble ML model (least absolute shrinkage and selection operator, kernel ridge, gradient boosting, elastic net, and light gradient boosting machine [LightGBM]) using data from 1583 applicants (2009-2020) to predict MCAT scores from cumulative GPA, core GPA, interview scores, merit points, and honors participation. The model was validated on 315 applicants admitted during the MCAT suspension (2021-2023), and its impact on admission rankings and performance correlations was evaluated. Results: Including the MCAT in the admissions composite did not meaningfully improve prediction of overall academic performance ( Conclusions: Within this single-institution study, MCAT scores provided limited incremental validity for predicting academic performance and reduced prediction of clinical skills. ML-based predictions of MCAT scores introduced sufficient error to affect admissions decisions, supporting a robust holistic admissions process during temporary MCAT disruptions while highlighting the need for validation across diverse institutional settings before broader generalization.