Evidence map›Paper›PMID 36737471›Full record

ArticleScientific reports2023

Evaluation of the portability of computable phenotypes with natural language processing in the eMERGE network.

Jennifer A Pacheco, Luke V Rasmussen, Ken Wiley, Thomas Nate Person, David J Cronkite, Sunghwan Sohn, Shawn Murphy, Justin H Gundelach, Vivian Gainer, Victor M Castro and 29 more

Abstract read
In one paragraph

Article in Scientific reports, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers.

0numbers the graph read from it
0cells of the map it votes in
14citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

14 citing papers in PubMed.

  1. The consensus-based CINEX guideline for reporting clinical information extraction studies.Journal of the American Medical Informatics Association : JAMIA · 2026
    Article
  2. Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. PhenoFit: a framework for determining computable phenotyping algorithm fitness for purpose and reuse.Journal of the American Medical Informatics Association : JAMIA · 2026
    Article
  9. Article
  10. Review
  11. Article
  12. Enhancing Cross-Domain Generalizability in Social Determinants of Health Extraction with Prompt-Tuning Large Language Models.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025
    Article
  13. Large language models facilitate the generation of electronic health record phenotyping algorithms.Journal of the American Medical Informatics Association : JAMIA · 2024
    Article
  14. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

39 authors.

Jennifer A PachecoNorthwestern University, Evanston, USA. japacheco@northwestern.edu.
Luke V RasmussenNorthwestern University, Evanston, USA.
Ken WileyNational Human Genome Research Institute, Bethesda, USA.
Thomas Nate PersonPennsylvania State University, Hershey, USA.
David J CronkiteKaiser Permanente Washington Health Research Institute, Seattle, USA.
Sunghwan SohnMayo Clinic, Rochester, USA.
Shawn MurphyMassachusetts General Hospital, Boston, USA.
Justin H GundelachMayo Clinic, Rochester, USA.
Vivian GainerMass General Brigham, Somerville, USA.
Victor M CastroMass General Brigham, Somerville, USA.
Cong LiuColumbia University, New York, USA.
Frank MentchChildren's Hospital of Philadelphia, Philadelphia, USA.
Todd LingrenCincinnati Children's Hospital Medical Center, Cincinnati, USA.
Agnes S SundaresanGeisinger, Danville, USA.
Garrett EickelbergNorthwestern University, Evanston, USA.
Valerie WillisNational Human Genome Research Institute, Bethesda, USA.
Al'ona FurmanchukNorthwestern University, Evanston, USA.
Roshan PatelGeisinger, Danville, USA.
David S CarrellKaiser Permanente Washington Health Research Institute, Seattle, USA.
Yu DengNorthwestern University, Evanston, USA.
Nephi WaltonIntermountain Healthcare, Salt Lake City, USA.
Benjamin A SatterfieldMayo Clinic, Rochester, USA.
Iftikhar J KulloMayo Clinic, Rochester, USA.
Ozan DikilitasMayo Clinic, Rochester, USA.
Joshua C SmithVanderbilt University Medical Center, Nashville, USA.
Josh F PetersonVanderbilt University Medical Center, Nashville, USA.
Ning ShangColumbia University, New York, USA.
Krzysztof KirylukColumbia University, New York, USA.
Yizhao NiCincinnati Children's Hospital Medical Center, Cincinnati, USA.
Yikuan LiNorthwestern University, Evanston, USA.
Girish N NadkarniIcahn School of Medicine at Mount Sinai, New York, USA.
Elisabeth A RosenthalUniversity of Washington, Seattle, USA.
Theresa L WalunasNorthwestern University, Evanston, USA.
Marc S WilliamsGeisinger, Danville, USA.
Elizabeth W KarlsonBrigham and Women's Hospital, Boston, USA.
Jodell E LinderVanderbilt University Medical Center, Nashville, USA.
Yuan LuoNorthwestern University, Evanston, USA.
Chunhua Weng *Columbia University, New York, USA.
WeiQi Wei *Vanderbilt University Medical Center, Nashville, USA.

Funding

The eMERGE Risk Assessment Network - Coordinating CenterU01HG011166 · NHGRI · VANDERBILT UNIVERSITY MEDICAL CENTER · PI Niall John Lennon, Joseph F. Peterson · 2020 to 2026
$16.8M
Genomic Basis of Susceptibility to COVID-19 Infection and its ComplicationsU01HG006379 · NHGRI · MAYO CLINIC ROCHESTER · PI Richard R. Sharp · 2011 to 2026
$16.5M
Finding Genomic Profiles of COVID-19 Phenotypes from the EHRU01HG008685 · NHGRI · BRIGHAM AND WOMEN'S HOSPITAL · PI ELIZABETH W KARLSON, Matthew S Lebo · 2015 to 2026
$13.6M
DNA Sequencing Support for the eMERGE NetworkU01HG008664 · NHGRI · BAYLOR COLLEGE OF MEDICINE · PI GIBBS, RICHARD A · 2015 to 2019
$10.9M
Utilizing Polygenic Risk to Understand and Improve Outcomes: A Model For Overturning Health Disparities Through Minority-Enriched Genomics HealthcareU01HG011175 · NHGRI · CHILDREN'S HOSP OF PHILADELPHIA · PI Hakon Hakonarson · 2020 to 2026
$10.3M
Northwestern Genomic Risk Assessment and Management ProgramU01HG011169 · NHGRI · NORTHWESTERN UNIVERSITY AT CHICAGO · PI REX L CHISHOLM, Elizabeth M McNally · 2020 to 2026
$7.4M
Polygenic Risk Scores for Healthier African American FamiliesU01HG011172 · NHGRI · CINCINNATI CHILDRENS HOSP MED CTR · PI Leah Claire Kottyan, LISA J MARTIN · 2020 to 2026
$7.2M
Vanderbilt Genome-Electronic Records (VGER) ProjectU01HG011181 · NHGRI · VANDERBILT UNIVERSITY MEDICAL CENTER · PI DAN M RODEN, Digna R Velez Edwards · 2020 to 2026
$7.0M
COVID and Translational Science supercomputer (CATS)S10OD030463 · OD · ICAHN SCHOOL OF MEDICINE AT MOUNT SINAI · PI KOVATCH, PATRICIA · 2021 to 2021
$2.0M
Big Omics Data Engine 2 SupercomputerS10OD026880 · OD · ICAHN SCHOOL OF MEDICINE AT MOUNT SINAI · PI KOVATCH, PATRICIA · 2019 to 2019
$2.0M
PheMAP: Measured, Automated Profile to Facilitate High Throughput PhenotypingR01GM139891 · NIGMS · VANDERBILT UNIVERSITY MEDICAL CENTER · PI WEI, WEI-QI · 2021 to 2024
$1.7M
NHGRI NIH HHS U01 HG006379NHGRI NIH HHS U01 HG008664NHGRI NIH HHS U01 HG008685NHGRI NIH HHS U01 HG011166NHGRI NIH HHS U01 HG011169NHGRI NIH HHS U01 HG011172NHGRI NIH HHS U01 HG011175NHGRI NIH HHS U01 HG011181NIAMS NIH HHS 5R21AR072262NIGMS NIH HHS R01 GM139891NIH HHS S10 OD026880NIH HHS S10 OD030463
6 · The paper itself

Abstract

The electronic Medical Records and Genomics (eMERGE) Network assessed the feasibility of deploying portable phenotype rule-based algorithms with natural language processing (NLP) components added to improve performance of existing algorithms using electronic health records (EHRs). Based on scientific merit and predicted difficulty, eMERGE selected six existing phenotypes to enhance with NLP. We assessed performance, portability, and ease of use. We summarized lessons learned by: (1) challenges; (2) best practices to address challenges based on existing evidence and/or eMERGE experience; and (3) opportunities for future research. Adding NLP resulted in improved, or the same, precision and/or recall for all but one algorithm. Portability, phenotyping workflow/process, and technology were major themes. With NLP, development and validation took longer. Besides portability of NLP technology and algorithm replicability, factors to ensure success include privacy protection, technical infrastructure setup, intellectual property agreement, and efficient communication. Workflow improvements can improve communication and reduce implementation time. NLP performance varied mainly due to clinical document heterogeneity; therefore, we suggest using semi-structured notes, comprehensive documentation, and customization options. NLP portability is possible with improved phenotype algorithm performance, but careful planning and architecture of the algorithms is essential to support local customizations.

Indexed as

Electronic Health RecordsNatural Language ProcessingAlgorithmsGenomicsPhenotype

Identifiers

PMID36737471
PMCPMC9898520

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.