Evidence map›Paper›PMID 38447593›Full record

ArticleJournal of the American Medical Informatics Association : JAMIA2024

Real world performance of the 21st Century Cures Act population-level application programming interface.

James R Jones, Daniel Gottlieb, Andrew J McMurry, Ashish Atreja, Pankaja M Desai, Brian E Dixon, Philip R O Payne, Anil J Saldanha, Prabhu Shankar, Yauheni Solad and 9 more

Abstract read
In one paragraph

Article in Journal of the American Medical Informatics Association : JAMIA, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers.

0numbers the graph read from it
0cells of the map it votes in
8citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

8 citing papers in PubMed.

  1. Article
  2. Review
  3. Review
  4. The health data utility and the resurgence of health information exchanges as a national resource.Journal of the American Medical Informatics Association : JAMIA · 2025
    Article
  5. Article
  6. Article
  7. Cumulus: A federated EHR-based learning system powered by FHIR and AI.medRxiv : the preprint server for health sciences · 2024
    Article
  8. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

19 authors.

James R JonesComputational Health Informatics Program, Boston Children's Hospital, Boston, MA 02215, United States.
Daniel GottliebComputational Health Informatics Program, Boston Children's Hospital, Boston, MA 02215, United States.
Andrew J McMurryComputational Health Informatics Program, Boston Children's Hospital, Boston, MA 02215, United States.
Ashish AtrejaDepartment of Health Innovation Technology, UC Davis Health, Rancho Cardova, CA 95670, United States.
Pankaja M DesaiDepartment of Internal Medicine, Rush University Medical Center, Chicago, IL 60612, United States.ORCID 0000-0002-0904-5905
Brian E DixonDepartment of Health Policy and Management, Fairbanks School of Public Health, Indiana University, Indianapolis, IN 46202, United States.ORCID 0000-0002-1121-0607
Philip R O PayneDepartment of Medicine, Washington University in St Louis, St Louis, MO 63110, United States.ORCID 0000-0002-9532-2998
Anil J SaldanhaDepartment of Health Innovation, Rush University Medical Center, Chicago, IL 60612, United States.
Prabhu ShankarDepartment of Health Innovation Technology, UC Davis Health, Rancho Cardova, CA 95670, United States.
Yauheni SoladDepartment of Health Innovation Technology, UC Davis Health, Rancho Cardova, CA 95670, United States.
Adam B WilcoxDepartment of Medicine, Washington University in St Louis, St Louis, MO 63110, United States.
Momeena S AliDepartment of Health Innovation Technology, UC Davis Health, Rancho Cardova, CA 95670, United States.
Eugene KangDepartment of Health Innovation Technology, UC Davis Health, Rancho Cardova, CA 95670, United States.
Andrew M MartinDepartment of Technical Services, Regenstrief Institute, Indianapolis, IN 46202, United States.
Elizabeth SprouseDouble Lantern Informatics, Atlanta, GA 30305, United States.
David E TaylorDepartment of Technical Services, Regenstrief Institute, Indianapolis, IN 46202, United States.
Michael TerryComputational Health Informatics Program, Boston Children's Hospital, Boston, MA 02215, United States.
Vladimir IgnatovComputational Health Informatics Program, Boston Children's Hospital, Boston, MA 02215, United States.
Kenneth D MandlComputational Health Informatics Program, Boston Children's Hospital, Boston, MA 02215, United States.ORCID 0000-0002-9781-0477

Funding

Instrumenting the Delivery System for a Genomics Research Information CommonsU01TR002623 · NCATS · BOSTON CHILDREN'S HOSPITAL · PI MANDL, KENNETH D. · 2019 to 2024
$8.6M
Translating Scientific Evidence into Practice using Digital Medicine and Electronic Patient Reported OutcomesU01TR002997 · NCATS · ICAHN SCHOOL OF MEDICINE AT MOUNT SINAI · PI ATREJA, ASHISH, RIZK, MAGED · 2020 to 2023
$4.7M
ACL HHS U18DP006500CDC HHSCDC HHS NU38OT000286CDC HHS U18DP006500NCATS NIH HHS U01 TR002623NCATS NIH HHS U01 TR002997NCCDPHP CDC HHS U18 DP006500OSTLTS CDC HHS NU38OT000286
6 · The paper itself

Abstract

objectiveTo evaluate the real-world performance of the SMART/HL7 Bulk Fast Health Interoperability Resources (FHIR) Access Application Programming Interface (API), developed to enable push button access to electronic health record data on large populations, and required under the 21st Century Cures Act Rule. MATERIALS AND

methodsWe used an open-source Bulk FHIR Testing Suite at 5 healthcare sites from April to September 2023, including 4 hospitals using electronic health records (EHRs) certified for interoperability, and 1 Health Information Exchange (HIE) using a custom, standards-compliant API build. We measured export speeds, data sizes, and completeness across 6 types of FHIR.

resultsAmong the certified platforms, Oracle Cerner led in speed, managing 5-16 million resources at over 8000 resources/min. Three Epic sites exported a FHIR data subset, achieving 1-12 million resources at 1555-2500 resources/min. Notably, the HIE's custom API outperformed, generating over 141 million resources at 12 000 resources/min. DISCUSSION: The HIE's custom API showcased superior performance, endorsing the effectiveness of SMART/HL7 Bulk FHIR in enabling large-scale data exchange while underlining the need for optimization in existing EHR platforms. Agility and scalability are essential for diverse health, research, and public health use cases.

conclusionTo fully realize the interoperability goals of the 21st Century Cures Act, addressing the performance limitations of Bulk FHIR API is critical. It would be beneficial to include performance metrics in both certification and reporting processes.

Indexed as

Health Information ExchangeHealth Level SevenDelivery of Health CareElectronic Health RecordsSoftwarecomputerizedhealth information interoperabilityhealth information systemsmedical informaticsmedical records systemspublic health informatics

Identifiers

PMID38447593
PMCPMC11031206

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.