Evidence map›Paper›PMID 33638343›Full record

ArticleJournal of the American Medical Informatics Association : JAMIA2021

Using machine learning to improve the accuracy of patient deterioration predictions: Mayo Clinic Early Warning Score (MC-EWS).

Santiago Romero-Brufau, Daniel Whitford, Matthew G Johnson, Joel Hickman, Bruce W Morlan, Terry Therneau, James Naessens, Jeanne M Huddleston

Open access · greenAbstract read
In one paragraph

Article in Journal of the American Medical Informatics Association : JAMIA, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
23citing papers in PubMed, 2 pooled it
6.6field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

23 citing papers in PubMed, 2 syntheses or guidelines pooled it, 64 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Review
  12. Review
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

8 authors at 4 institutions in 1 country.

Santiago Romero-BrufauDepartment of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, USA.ORCID 0000-0002-9922-0083
Daniel WhitfordDepartment of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, USA.
Matthew G JohnsonDepartment of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA.
Joel HickmanDepartment of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA.
Bruce W MorlanDepartment of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA.
Terry TherneauDepartment of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA.
James NaessensDepartment of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA.
Jeanne M HuddlestonDepartment of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, USA.
Mayo Clinic in Florida · USHarvard University · USMayo Clinic · USUniversity of Minnesota · US

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectiveWe aimed to develop a model for accurate prediction of general care inpatient deterioration. MATERIALS AND

methodsTraining and internal validation datasets were built using 2-year data from a quaternary hospital in the Midwest. Model training used gradient boosting and feature engineering (clinically relevant interactions, time-series information) to predict general care inpatient deterioration (resuscitation call, intensive care unit transfer, or rapid response team call) in 24 hours. Data from a tertiary care hospital in the Southwest were used for external validation. C-statistic, sensitivity, positive predictive value, and alert rate were calculated for different cutoffs and compared with the National Early Warning Score. Sensitivity analysis evaluated prediction of intensive care unit transfer or resuscitation call.

resultsTraining, internal validation, and external validation datasets included 24 500, 25 784 and 53 956 hospitalizations, respectively. The Mayo Clinic Early Warning Score (MC-EWS) demonstrated excellent discrimination in both the internal and external validation datasets (C-statistic = 0.913, 0.937, respectively), and results were consistent in the sensitivity analysis (C-statistic = 0.932 in external validation). At a sensitivity of 73%, MC-EWS would generate 0.7 alerts per day per 10 patients, 45% less than the National Early Warning Score. DISCUSSION: Low alert rates are important for implementation of an alert system. Other early warning scores developed for the general care ward have achieved lower discrimination overall compared with MC-EWS, likely because MC-EWS includes both nursing assessments and extensive feature engineering.

conclusionsMC-EWS achieved superior prediction of general care inpatient deterioration using sophisticated feature engineering and a machine learning approach, reducing alert rate.

Indexed as

Early Warning ScoreHospitalizationHumansInpatientsIntensive Care UnitsMachine Learningclinical deteriorationearly warning scoremachine learning

Identifiers

PMID33638343
PMCPMC8661441
OpenAlexW3131760363

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.