Evidence map›Paper›PMID 30239810›Full record

SynthesisJournal of the American Medical Informatics Association : JAMIA2018

Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review.

Emily M Powers, Richard N Shiffman, Edward R Melnick, Andrew Hickner, Mona Sharifi

Abstract readSystematic Review
In one paragraph

Synthesis in Journal of the American Medical Informatics Association : JAMIA, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 67 papers, 4 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
67citing papers in PubMed, 4 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

67 citing papers in PubMed, 4 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Trial
  6. Interruptive Electronic Alerts for Choosing Wisely Recommendations: A Cluster Randomized Controlled Trial.Journal of the American Medical Informatics Association : JAMIA · 2022
    Trial
  7. Trial
  8. Trial
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Review
  15. Article
  16. Article
  17. Review
  18. Article
  19. Article
  20. Article

7 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

5 authors.

Emily M PowersYale Center for Medical Informatics, Yale University School of Medicine, New Haven, Connecticut, USA.
Richard N ShiffmanYale Center for Medical Informatics, Yale University School of Medicine, New Haven, Connecticut, USA.
Edward R MelnickYale Center for Medical Informatics, Yale University School of Medicine, New Haven, Connecticut, USA.
Andrew HicknerCushing/Whitney Medical Library, Yale University School of Medicine, New Haven, Connecticut, USA.
Mona SharifiYale Center for Medical Informatics, Yale University School of Medicine, New Haven, Connecticut, USA.

Funding

MEDICAL INFORMATICS RESEARCH TRAINING AT YALET15LM007056 · NLM · YALE UNIVERSITY · PI Mark Bender Gerstein, LUCILA OHNO-MACHADO · 1987 to 2026
$22.2M
Clinical Decision Support for Mild Traumatic Brain InjuryK08HS021271 · AHRQ · YALE UNIVERSITY · PI MELNICK, EDWARD ROBERT · 2013 to 2017
$772k
Using Electronic Health Records to Support Decision-Making in Pediatric Obesity CareK08HS024332 · AHRQ · YALE UNIVERSITY · PI SHARIFI, MAHNOOSH · 2016 to 2018
$461k
AHRQ HHS K08 HS021271AHRQ HHS K08 HS024332NLM NIH HHS T15 LM007056
6 · The paper itself

Abstract

Objective: Clinical decision support (CDS) hard-stop alerts-those in which the user is either prevented from taking an action altogether or allowed to proceed only with the external override of a third party-are increasingly common but can be problematic. To understand their appropriate application, we asked 3 key questions: (1) To what extent are hard-stop alerts effective in improving patient health and healthcare delivery outcomes? (2) What are the adverse events and unintended consequences of hard-stop alerts? (3) How do hard-stop alerts compare to soft-stop alerts? Methods and Materials: Studies evaluating computerized hard-stop alerts in healthcare settings were identified from biomedical and computer science databases, gray literature sites, reference lists, and reviews. Articles were extracted for process outcomes, health outcomes, unintended consequences, user experience, and technical details. Results: Of 32 studies, 15 evaluated health outcomes, 16 process outcomes only, 10 user experience, and 4 compared hard and soft stops. Seventy-nine percent showed improvement in health outcomes and 88% in process outcomes. Studies reporting good user experience cited heavy user involvement and iterative design. Eleven studies reported on unintended consequences including avoidance of hard-stopped workflow, increased alert frequency, and delay to care. Hard stops were superior to soft stops in 3 of 4 studies. Conclusions: Hard stops can be effective and powerful tools in the CDS armamentarium, but they must be implemented judiciously with continuous user feedback informing rapid, iterative design. Investigators must report on associated health outcomes and unintended consequences when implementing IT solutions to clinical problems.

Indexed as

Decision Support Systems, ClinicalMedical Records Systems, ComputerizedTherapy, Computer-AssistedElectronic Health RecordsHumansMedical Order Entry Systems

Identifiers

PMID30239810
PMCPMC6915824

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.