Cross-Site Predictions of Readmission After Psychiatric Hospitalization With Mood or Psychotic Disorders: Retrospective Study.

Background: Patients with mood or psychotic disorders experience high rates of unplanned hospital readmissions. Predicting the likelihood of readmission can guide discharge decisions and optimize patient care. Objective: The purpose of this study is to evaluate the predictive power of structured variables from electronic health records for all-cause readmission across multiple sites within the Mass General Brigham health system and to assess the transportability of prediction models between sites. Methods: This retrospective, multisite study analyzed structured variables from electronic health records separately for each site to develop in-site prediction models. The transportability of these models was evaluated by applying them across different sites. Predictive performance was measured using the F1-score, and additional adjustments were made to account for differences in predictor distributions. Results: The study found that the relevant predictors of readmission varied significantly across sites. For instance, length of stay was a strong predictor at only 3 of the 4 sites. In-site prediction models achieved an average F1-score of 0.661, whereas cross-site predictions resulted in a lower average F1-score of 0.616. Efforts to improve transportability by adjusting for differences in predictor distributions did not improve performance. Conclusions: The findings indicate that individual site-specific models are necessary to achieve reliable prediction accuracy. Furthermore, the results suggest that the current set of predictors may be insufficient for cross-site model transportability, highlighting the need for more advanced predictor variables and predictive algorithms to gain robust insights into the factors influencing early psychiatric readmissions.