Feasibility of AI and Human Standardized Patients to Enhance Customer Discovery Communication Skills in Medical Students: Preliminary Evaluation of an Observational Cohort Study.

backgroundMedical students must build skills beyond traditional clinical domains to best shape the evolving health care system and fill a variety of professional roles after graduation. AI offers new methods for medical education.

objectiveThis study aimed to develop and evaluate the feasibility and preliminary effectiveness of traditional (human) standardized patient (SP) engagement and generative AI (GenAI)-generated SP engagement as a low-stakes way for students to practice customer discovery interviews.

methodsAn interactive classroom experience was created in the UMass Chan Medical School interprofessional Center for Experiential Learning and Simulation (iCELS) using an observational cohort design to simulate 2 different interview scenarios with human SPs and AI-generated SPs. The simulation with human SPs was conducted with 74 first-year medical students in the Entrepreneurship, Biodesign, and Innovation Pathway starting in 2023. In 2026, 2 GenAI chatbot interview agents, using OpenAI GPT-5.2, were developed and added as an additional part of the simulation experience. The 2 scenario-specific GenAI agents (patient and clinician/physician) used structured instructions defining their roles, interview contexts, knowledge boundaries, response styles, time limits, feedback, and scoring rubric. The simulations took place over a single 2-hour session in the spring semester. Interview scenarios, sample questions and answers, and an evaluation rubric were developed for both SPs to promote consistency. Primary outcomes were student attitudes regarding the acceptability of the simulation and self-efficacy, collected via a validated online survey immediately after class. Students were asked to rate statements on a 3-point (1="not at all relevant," 2="somewhat relevant," and 3="very relevant"), 4-point (1="strongly disagree" to 4="strongly agree"), or 5-point (1="strongly disagree," 3="neutral," and 5="strongly agree") Likert scale and were also allowed to provide open-ended comments.

resultsThe students gave the simulation with human SPs high scores, with 85% (23/27) agreeing or strongly agreeing that the exercise met learning objectives, with a median score of 4 (IQR 1) on a 4-point Likert scale. Responses to the AI chatbot session had a bimodal distribution; for example, 65% (13/20) of students agreed that "This experience with an AI chatbot SP helped me practice my customer discovery interview skills" while 35% (7/20) disagreed with this statement. A Mann-Whitney U test compared responses between the human SP and AI chatbot groups and revealed no significant difference in responses.

conclusionsThis preliminary study with an AI chatbot was able to replicate realistic customer discovery interviews in 2 different scenarios. Larger studies are needed to determine the feasibility and acceptability of these chatbots for practicing customer discovery skills. Future work will create transcripts of both the AI chatbot and human SP interviews and use independent raters to score interview quality based on our scoring rubric. Finally, we plan to enhance the chatbots to provide a more immersive and realistic experience for students.