Validation of an Electronic Health Record-Based Machine Learning Model Compared With Clinical Risk Scores for Gastrointestinal Bleeding.

BACKGROUND &

aimsGuidelines recommend use of risk stratification scores for patients presenting with gastrointestinal bleeding (GIB) to identify very-low-risk patients eligible for discharge from emergency departments. Machine learning models may outperform existing scores and can be integrated within the electronic health record (EHR) to provide real-time risk assessment without manual data entry. We present the first EHR-based machine learning model for GIB.

methodsThe training cohort comprised 2546 patients and internal validation of 850 patients presenting with overt GIB (ie, hematemesis, melena, and hematochezia) to emergency departments of 2 hospitals from 2014 to 2019. External validation was performed on 926 patients presenting to a different hospital with the same EHR from 2014 to 2019. The primary outcome was a composite of red blood cell transfusion, hemostatic intervention (ie, endoscopic, interventional radiologic, or surgical), and 30-day all-cause mortality. We used structured data fields in the EHR, available within 4 hours of presentation, and compared the performance of machine learning models with current guideline-recommended risk scores, Glasgow-Blatchford Score, and Oakland Score. Primary analysis was area under the receiver operating characteristic curve. Secondary analysis was specificity at 99% sensitivity to assess the proportion of patients correctly identified as very low risk.

resultsThe machine learning model outperformed the Glasgow-Blatchford Score (area under the receiver operating characteristic curve, 0.92 vs 0.89; P < .001) and Oakland Score (area under the receiver operating characteristic curve, 0.92 vs 0.89; P < .001). At the very-low-risk threshold of 99% sensitivity, the machine learning model identified more very-low-risk patients: 37.9% vs 18.5% for Glasgow-Blatchford Score and 11.7% for Oakland Score (P < .001 for both comparisons).

conclusionsAn EHR-based machine learning model performs better than currently recommended clinical risk scores and identifies more very-low-risk patients eligible for discharge from the emergency department.