Evidence map›Paper›PMID 29908902›Full record

ArticleJournal of biomedical informatics2018

A study of generalizability of recurrent neural network-based predictive models for heart failure onset risk using a large and heterogeneous EHR data set.

Laila Rasmy, Yonghui Wu, Ningtao Wang, Xin Geng, W Jim Zheng, Fei Wang, Hulin Wu, Hua Xu, Degui Zhi

Abstract read
In one paragraph

Article in Journal of biomedical informatics, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 51 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
51citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

51 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Article
  4. Article
  5. Evaluation of trajectory analysis for disease risk assessment: a scoping review.Journal of the American Medical Informatics Association : JAMIA · 2026
    Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Review
  13. Article
  14. Article
  15. Article
  16. Review
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors.

Laila RasmySchool of Biomedical Informatics, University of Texas Health Science Center at Houston (UTHealth), Houston, TX, United States.
Yonghui WuDepartment of Health Outcomes & Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States.
Ningtao WangDepartment of Biostatistics and Data Science, School of Public Health, University of Texas Health Science Center at Houston (UTHealth), Houston, TX, United States.
Xin GengBGI-Shenzhen, Shenzhen, 518083, China.
W Jim ZhengSchool of Biomedical Informatics, University of Texas Health Science Center at Houston (UTHealth), Houston, TX, United States.
Fei WangDepartment of Healthcare Policy and Research, Weill Cornell Medicine, Cornell University, New York, NY, United States.
Hulin WuDepartment of Biostatistics and Data Science, School of Public Health, University of Texas Health Science Center at Houston (UTHealth), Houston, TX, United States.
Hua XuSchool of Biomedical Informatics, University of Texas Health Science Center at Houston (UTHealth), Houston, TX, United States.
Degui ZhiSchool of Biomedical Informatics, University of Texas Health Science Center at Houston (UTHealth), Houston, TX, United States. Electronic address: Degui.Zhi@uth.tmc.edu.

Funding

Next-Generation Bioinformatics for Next-Generation SequencingR01HG008115 · NHGRI · BAYLOR COLLEGE OF MEDICINE · PI YU, FULI, ZHANG, KUI · 2014 to 2016
$1.4M
NHGRI NIH HHS R01 HG008115
6 · The paper itself

Abstract

Recently, recurrent neural networks (RNNs) have been applied in predicting disease onset risks with Electronic Health Record (EHR) data. While these models demonstrated promising results on relatively small data sets, the generalizability and transferability of those models and its applicability to different patient populations across hospitals have not been evaluated. In this study, we evaluated an RNN model, RETAIN, over Cerner Health Facts® EMR data, for heart failure onset risk prediction. Our data set included over 150,000 heart failure patients and over 1,000,000 controls from nearly 400 hospitals. Convincingly, RETAIN achieved an AUC of 82% in comparison to an AUC of 79% for logistic regression, demonstrating the power of more expressive deep learning models for EHR predictive modeling. The prediction performance fluctuated across different patient groups and varied from hospital to hospital. Also, we trained RETAIN models on individual hospitals and found that the model can be applied to other hospitals with only about 3.6% of reduction of AUC. Our results demonstrated the capability of RNN for predictive modeling with large and heterogeneous EHR data, and pave the road for future improvements.

Indexed as

Deep LearningElectronic Health RecordsNeural Networks, ComputerAgedAged, 80 and overAlgorithmsArea Under CurveCase-Control StudiesComputer SimulationDatabases, FactualFemaleHeart FailureHumansLogistic ModelsMaleMedical InformaticsDeep learningEHRPredictive modelingRNN

Identifiers

PMID29908902
PMCPMC6076336

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.