Application of machine learning models in predicting the risk of thromboembolic events in patients with nonvariceal gastrointestinal bleeding.

backgroundClinically, patients with nonvariceal gastrointestinal bleeding (NVGB) are prone to thromboembolic events, but the specific risk remains unclear.

aimTo identify risk factors and evaluate the performance of five machine learning (ML) models in predicting the risk of thromboembolic events in patients with NVGB.

methodsThis retrospective cohort study enrolled 866 patients from a tertiary hospital for model training and internal validation, and 282 patients from three other tertiary hospitals for external validation. These data were used to develop five ML models to predict the risk of thromboembolic events in patients with NVGB. After initial feature selection by training ML models, ten variables were selected to construct simplified ML models. Model performance was evaluated using accuracy, precision, sensitivity, specificity, F1-score and area under the receiver operating characteristic curve. Calibration curve and decision curve analysis were used to further evaluate the predicted probabilities and net benefits of the models.

resultsDuring hospitalization, the incidence of thromboembolic events was 25.61% in patients with NVGB. The categorical boosting (CatBoost) algorithm which combined variable importance and SHapley Additive exPlanations values identified 10 independent predictors of thromboembolic events: (1) History of anticoagulant drug use; (2) D-dimer level; (3) Age; (4) History of thromboembolism; (5) Length of hospital stays; (6) Intensive care unit (ICU) admission; (7) Hemoglobin level; (8) Use of hemostatic drugs; (9) Heart rate; and (10) Serum albumin level. We developed five simplified ML prediction models (L1 regularized logistic regression, random forest, support vector machines, extreme gradient boosting, and CatBoost) based on the above 10 predictors, which achieved area under the receiver operating characteristic curves of 0.805, 0.804, 0.806, 0.746, and 0.815 in external validation, respectively. The performance of all five ML models significantly exceeded that of D-dimer alone in both internal and external validation. The CatBoost model demonstrated good calibration and accuracy, achieving the lowest Brier score of 0.131 and 0.110 in the internal and external validation set, respectively. Of the five models, the CatBoost model was considered the preferred choice in clinical settings.

conclusionThe findings in this study enable effective and timely preventive interventions for high-risk patients, and help avoid unnecessary monitoring in low-risk patients.