ArticleJournal of biomedical informatics2025
Conceptual framework for prediction models of patient deterioration based on nursing documentation patterns: reproducibility and generalizability with a large number of hospitals across the United States.
Article in Journal of biomedical informatics, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
1 citing paper in PubMed.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
Abstract
objectiveThe Health Process Model (HPM)-ExpertSignals Conceptual Framework posits that healthcare professionals' patient care behaviors can be used to predict in-hospital deterioration. Prediction models based on this framework have been validated using data from 4 hospitals within two healthcare systems. As clinician-system interactions may differ across organizations, this study aimed to evaluate the reproducibility and generalizability of the underlying conceptual framework using data from over 200 hospitals across the US.
methodsThis study used eICU-CRD, a publicly accessible dataset with data from 208 US hospitals. A logistic regression model was developed to predict in-hospital deterioration following the HPM-ExpertSignals conceptual framework. To test its reproducibility, patients were randomly split into training and testing datasets. After bootstrap testing of the model, the mean precision-recall curve (AUPRC) was compared with outcomes from previously published studies. For generalizability testing, the hospitals in the dataset were randomly assigned into model training or testing sets. After the model was trained with training hospitals' data, generalizability was measured as the percentage of testing hospitals with an AUPRC at or above a baseline performance obtained in the reproducibility experiment.
resultsThe AUPRC in the reproducibility experiment was 0.10 (0.09,0.11; 95% CI), equivalent to the AUPRC reported in a previous study at 0.093 (0.09, 0.096; 95% CI). In the generalizability experiment, 94% of the testing hospitals had AUPRC at or above the baseline AUPRC of 0.10.
conclusionThe study provides evidence supporting the reproducibility of a predictive model following the HPM-ExpertSignals framework. This model also generalized to most hospitals without additional training. Nevertheless, some hospitals still obtained lower-than-expected performance, highlighting the need for model evaluation and potential fine-tuning before local adoption. Similar studies are needed to investigate the reproducibility and generalizability of other classes of machine learning models in healthcare.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.