When Artificial Intelligence Fails the Future: How Unchecked Artificial Intelligence Could Amplify Inequality in Medical Education.

Background: Artificial intelligence is influencing healthcare but carries potential biases from historically trained data. This study was the first to examine racial and gender bias in large language model architecture applied to medical student career advising. Methods: Two hundred synthetic medical student profiles were created with systematically varied demographic and academic characteristics. Each profile was evaluated using a standardized advising prompt that generated the top 3 specialty recommendations. Specialties were assigned competitiveness scores based on National Resident Matching Program data. Univariate and multivariate analyses assessed associations between applicant characteristics and predicted competitiveness and surgical specialty recommendations. A compensatory scoring analysis estimated the additional USMLE Step 2 Clinical Knowledge points required for applicants from different demographic groups to achieve competitiveness equivalent to a White male reference profile. Results: Male applicants were rated more competitive than female applicants ( Conclusions: Our findings demonstrated that, if left unchecked, large language models such as ChatGPT perpetuate racial and gender biases when advising medical students, amplifying historical inequities in medicine.