Evidence map›Paper›PMID 38154472›Full record

ArticleApplied clinical informatics2024

Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record.

Kyle Nolla, Luke V Rasmussen, Nan E Rothrock, Zeeshan Butt, Michael Bass, Kristina Davis, David Cella, Richard Gershon, Cynthia Barnard, Ryan Chmiel and 5 more

Open access · hybridAbstract read
In one paragraph

Article in Applied clinical informatics, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers.

0numbers the graph read from it
0cells of the map it votes in
11citing papers in PubMed
2.3field-weighted citation impact, top 10% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

11 citing papers in PubMed, 9 citations in OpenAlex.

  1. Trial
  2. Article
  3. Using confirmatory factor analysis to identify symptom clusters in individuals with acquired disabilities.Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2026
    Article
  4. Article
  5. A Surgeon's Field Guide to Value-Based Specialty Care.Laryngoscope investigative otolaryngology · 2025
    Review
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Enhancing Measurement Precision of Patient-Reported Outcomes Using Item Response Theory.Inquiry : a journal of medical care organization, provision and financing
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors at 3 institutions in 1 country.

Kyle NollaDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.ORCID 0000-0002-8481-4655
Luke V RasmussenDepartment of Preventative Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.ORCID 0000-0002-4497-8049
Nan E RothrockDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.ORCID 0000-0002-4821-5030
Zeeshan ButtPhreesia, Inc, Clinical Content, Wilmington, DE, USA.
Michael BassDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.ORCID 0000-0002-7732-6557
Kristina DavisDepartment of Nursing Quality, Stanford Health Care, Stanford, California, United States.ORCID 0000-0002-3820-7424
David CellaDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.ORCID 0000-0002-9881-4541
Richard GershonDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.ORCID 0000-0003-0085-0112
Cynthia BarnardDepartment of General Internal Medicine, Feinberg School of Medicine, Northwestern University and Northwestern Memorial HealthCare, Chicago, Illinois, United States.ORCID 0000-0002-5522-2407
Ryan ChmielDepartment of Information Services, Northwestern Memorial HealthCare, Chicago, Illinois, United States.
Federico AlmarazDepartment of Information Services, Northwestern Memorial HealthCare, Chicago, Illinois, United States.
Michael SchachterDepartment of Information Services, Northwestern Memorial HealthCare, Chicago, Illinois, United States.
Therese NelsonClinical and Translational Sciences Institute, Northwestern University, Chicago, Illinois, United States.ORCID 0000-0001-6390-6430
Michelle LangerDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.
Justin StarrenDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States.
Northwestern University · USNorthwestern Medicine · USStanford Health Care · US

Funding

Northwestern University Clinical and Translational Science Institute (NUCATS)UL1TR001422 · NCATS · NORTHWESTERN UNIVERSITY AT CHICAGO · PI D'AQUILA, RICHARD · 2015 to 2023
$56.8M
Multidisciplinary Clinical and Translational Science (MCTS) Program (UL1)UL1TR000150 · NCATS · NORTHWESTERN UNIVERSITY AT CHICAGO · PI LLOYD-JONES, DONALD M · 2012 to 2013
$10.1M
Improving Patient Reported Outcome Data for Research through Seamless Integration of the PROMIS Toolkit into EHR WorkflowsU01TR001806 · NCATS · NORTHWESTERN UNIVERSITY AT CHICAGO · PI STARREN, JUSTIN B. · 2016 to 2019
$6.3M
Cancer Prevention, Control, and Survivorship Research Training ProgramT32CA193193 · NCI · NORTHWESTERN UNIVERSITY AT CHICAGO · PI Brian L Hitsman, BETINA YANEZ · 2015 to 2026
$3.8M
NCATS NIH HHS U01 TR001806NCATS NIH HHS U01TR001806NCATS NIH HHS UL1 TR000150NCATS NIH HHS UL1 TR001422NCI NIH HHS T32 CA193193
6 · The paper itself

Abstract

backgroundPatient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR.

objectivesThis paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic.

methodsThe overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software.

resultsThe middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system.

conclusionThis project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Indexed as

ComputersElectronic Health RecordsHumansPatient Reported Outcome MeasuresSoftware

Identifiers

PMID38154472
PMCPMC10881259
OpenAlexW4390315312

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.