Adapting the TeamSTEPPS team performance observation tool for dyadic interprofessional VR simulations (vTPOT): a multi-step validation study.

backgroundInterprofessional teamwork is a key determinant of patient safety in medical emergencies, yet its structured assessment remains resource-intensive and is rarely integrated into undergraduate curricula. Immersive virtual reality (VR) offers scalable environments for training and assessing teamwork but introduces contextual constraints such as dyadic team structures, avatar-mediated interaction, and limited non-verbal cues. Established assessment tools, including the Team Performance Observation Tool (TPOT), have not been systematically adapted or validated for these VR-specific conditions.

methodsWe adapted the TPOT for dyadic VR-based emergency simulations (vTPOT) through expert-driven item revision, development of behavioral anchors, and a two-round modified Delphi process with 15 interprofessional experts. No new VR-specific items were introduced, instead, items that could not be meaningfully observed in immersive VR were removed or consolidated to preserve conceptual stability. Five trained raters independently evaluated 21 VR simulation videos from interprofessional dyads (medical and nursing students) using both the original TPOT and the vTPOT. Following Kane’s argument-based validation framework, we examined evidence for scoring (content validity, association with the original TPOT), generalization (internal consistency, interrater reliability), extrapolation (associations with teamwork perceptions and objective medical performance), and implications (written qualitative responses from raters) on potential unintended effects and implications for feedback and assessment).

resultsThe final vTPOT comprises 15 behaviorally anchored items across five teamwork domains. Internal consistency was high (Cronbach’s α = 0.92), and interrater reliability was moderate to substantial (intraclass correlation coefficient = 0.68). vTPOT scores showed strong convergence with the original TPOT (r = 0.92) and with participants’ teamwork perceptions (T-TPQ; ρ = 0.64 and 0.80 for nursing and medical students, respectively). Higher vTPOT scores were positively associated with objectively measured medical performance in VR scenarios (r = 0.54). Raters reported that the vTPOT supported structured observation and feedback, while also highlighting context-specific limitations and potential unintended effects.

conclusionsThe vTPOT appears to be a feasible and reliable instrument for assessing interprofessional teamwork in dyadic VR simulations within the studied context. Initial validity evidence supports its intended use for low-stakes assessment and research. Further studies are needed to examine construct structure and extend validation to additional contexts, populations, and outcomes to strengthen the overall validity argument.