Evidence map›Paper›PMID 33176294›Full record

ArticlePhysiological measurement2021

Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020.

Erick A Perez Alday, Annie Gu, Amit J Shah, Chad Robichaux, An-Kwok Ian Wong, Chengyu Liu, Feifei Liu, Ali Bahrami Rad, Andoni Elola, Salman Seyedi and 4 more

Abstract read
In one paragraph

Article in Physiological measurement, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 84 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
84citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

84 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Discovering Interpretable Semantics from Radio Signals for Contactless Cardiac Monitoring.Advanced science (Weinheim, Baden-Wurttemberg, Germany) · 2026
    Article
  10. Review
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article

24 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

14 authors.

Erick A Perez AldayDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.ORCID 0000-0002-4269-0965
Annie GuDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.
Amit J ShahDepartment of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, United States of America.ORCID 0000-0001-9099-9687
Chad RobichauxDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.
An-Kwok Ian WongDepartment of Medicine, Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, Emory University, Atlanta, GA, United States of America.
Chengyu LiuSchool of Instrument Science and Engineering, Southeast University, Nanjing, Jiangsu, People's Republic of China.
Feifei LiuSchool of Science, Shandong Jianzhu University, Jinan, Shandong, People's Republic of China.
Ali Bahrami RadDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.
Andoni ElolaDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.ORCID 0000-0003-4401-7376
Salman SeyediDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.
Qiao LiDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.
Ashish SharmaDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.
Gari D CliffordDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.ORCID 0000-0002-5709-201X
Matthew A ReynaDepartment of Biomedical Informatics, Emory University, Atlanta, GA, United States of America.ORCID 0000-0003-4688-7965

Funding

Implementing a Maternal health and PRegnancy Outcomes Vision for Everyone (IMPROVE)UL1TR002378 · NCATS · EMORY UNIVERSITY · PI Andres J Garcia, Elizabeth O. Ofili · 2017 to 2026
$92.1M
Research Resource for Complex Physiologic SignalsR01EB030362 · NIBIB · BETH ISRAEL DEACONESS MEDICAL CENTER · PI GOLDBERGER, ARY LOUIS · 2020 to 2025
$5.8M
Research Resource for Complex Physiologic SignalsR01GM104987 · NIGMS · BETH ISRAEL DEACONESS MEDICAL CENTER · PI GOLDBERGER, ARY LOUIS, MARK, ROGER GREENWOOD · 2012 to 2019
$5.8M
Critical Care Training ProgramT32GM095442 · NIGMS · EMORY UNIVERSITY · PI Craig M Coopersmith · 2011 to 2026
$4.7M
Emotional Stress as a Risk Factor for Arrhythmia: Ischemic and Genetic MechanismsK23HL127251 · NHLBI · EMORY UNIVERSITY · PI SHAH, AMIT JASVANT · 2016 to 2019
$749k
NCATS NIH HHS UL1 TR002378NHLBI NIH HHS K23 HL127251NIBIB NIH HHS R01 EB030362NIGMS NIH HHS R01 GM104987NIGMS NIH HHS T32 GM095442
6 · The paper itself

Abstract

objectiveVast 12-lead ECGs repositories provide opportunities to develop new machine learning approaches for creating accurate and automatic diagnostic systems for cardiac abnormalities. However, most 12-lead ECG classification studies are trained, tested, or developed in single, small, or relatively homogeneous datasets. In addition, most algorithms focus on identifying small numbers of cardiac arrhythmias that do not represent the complexity and difficulty of ECG interpretation. This work addresses these issues by providing a standard, multi-institutional database and a novel scoring metric through a public competition: the PhysioNet/Computing in Cardiology Challenge 2020. APPROACH: A total of 66 361 12-lead ECG recordings were sourced from six hospital systems from four countries across three continents; 43 101 recordings were posted publicly with a focus on 27 diagnoses. For the first time in a public competition, we required teams to publish open-source code for both training and testing their algorithms, ensuring full scientific reproducibility. MAIN

resultsA total of 217 teams submitted 1395 algorithms during the Challenge, representing a diversity of approaches for identifying cardiac abnormalities from both academia and industry. As with previous Challenges, high-performing algorithms exhibited significant drops ([Formula: see text]10%) in performance on the hidden test data. SIGNIFICANCE: Data from diverse institutions allowed us to assess algorithmic generalizability. A novel evaluation metric considered different misclassification errors for different cardiac abnormalities, capturing the outcomes and risks of different diagnoses. Requiring both trained models and code for training models improved the generalizability of submissions, setting a new bar in reproducibility for public data science competitions.

Indexed as

CardiologyElectrocardiographyAlgorithmsArrhythmias, CardiacDatabases, FactualFemaleHumansMaleMiddle AgedReproducibility of Results

Identifiers

PMID33176294
PMCPMC8015789

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.