Observational studyBMJ open2019
Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalised general internal medicine patients: a prospective observational study.
Observational study in BMJ open, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT02648828 (Understanding Physician Signout Risk Perception), which is not on this map. Cited by 17 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Understanding Physician Signout Risk Perception
Who cites it
17 citing papers in PubMed, 1 synthesis or guideline pooled it, 31 citations in OpenAlex.
- Accuracy of the large language model ChatGPT in adult emergency department triage: a systematic review and meta-analysis.BMC emergency medicine · 2026Pooled it
- Assessing Risk of Early ICU Admission or Death at Emergency Department Triage: Clinical Judgment Versus Early Warning Scores.Academic emergency medicine : official journal of the Society for Academic Emergency Medicine · 2026Observational
- Multi-Evidence Clinical Reasoning With Retrieval-Augmented Generation for Emergency Triage: Retrospective Evaluation Study.JMIR medical informatics · 2026Article
- Early prediction of in-hospital deterioration after emergency department admission using machine learning models.BMC emergency medicine · 2026Article
- Development and Validation of a NEWS2-Enhanced Multivariable Prediction Model for Clinical Deterioration and In-Hospital Mortality in Hospitalized Adults.Medicina (Kaunas, Lithuania) · 2025Article
- Deep Learning-Based Electrocardiogram Model (EIANet) to Predict Emergency Department Cardiac Arrest: Development and External Validation Study.Journal of medical Internet research · 2025Article
- Key enablers and barriers to ICU nurses' professional identity: a qualitative study.Frontiers in medicine · 2025Article
- Leading with AI in critical care nursing: challenges, opportunities, and the human factor.BMC nursing · 2024Article
- Exploiting the Features of Clinical Judgment to Improve Assessment of Disease Severity in the Emergency Department: An Acutelines Study.Journal of clinical medicine · 2024Article
- Developing and Validating a Prediction Model For Death or Critical Illness in Hospitalized Adults, an Opportunity for Human-Computer Collaboration.Critical care explorations · 2023Article
- Artificial Intelligence in Resuscitation: A Scoping Review.Journal of clinical medicine · 2023Article
- Entering the new digital era of intensive care medicine: an overview of interdisciplinary approaches to use artificial intelligence for patients' benefit.European journal of anaesthesiology and intensive care · 2023Article
- Comparing In-Hospital Mortality Prediction by Senior Emergency Resident's Judgment and Prognostic Models in the Emergency Department.BioMed research international · 2023Article
- Article
- Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting.Journal of medical Internet research · 2021Review
- Performance Analysis of the National Early Warning Score and Modified Early Warning Score in the Adaptive COVID-19 Treatment Trial Cohort.Critical care explorations · 2021Article
- Detecting Deteriorating Patients in the Hospital: Development and Validation of a Novel Scoring System.American journal of respiratory and critical care medicine · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors at 3 institutions in 1 country.
Funding
Abstract
objectiveOur study compares physician judgement with an automated early warning system (EWS) for predicting clinical deterioration of hospitalised general internal medicine patients.
designProspective observational study of clinical predictions made at the end of the daytime work-shift for an academic general internal medicine floor team compared with the risk assessment from an automated EWS collected at the same time.
settingInternal medicine teaching wards at a single tertiary care academic medical centre in the USA.
participantsIntern physicians working on the internal medicine wards and an automated EWS (Rothman Index by PeraHealth). OUTCOME: Clinical deterioration within 24 hours including cardiac or pulmonary arrest, rapid response team activation or unscheduled intensive care unit transfer.
resultsWe collected predictions for 1874 patient days and saw 35 clinical deteriorations (1.9%). The area under the receiver operating curve (AUROC) for the EWS was 0.73 vs 0.70 for physicians (p=0.571). A linear regression model combining physician and EWS predictions had an AUROC of 0.75, outperforming physicians (p=0.016) and the EWS (p=0.05).
conclusionsThere is no significant difference in the performance of the EWS and physicians in predicting clinical deterioration at 24 hours on an inpatient general medicine ward. A combined model outperformed either alone. The EWS and physicians identify partially overlapping sets of at-risk patients suggesting they rely on different cues or decision rules for their predictions. TRIAL REGISTRATION NUMBER: NCT02648828.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.