Evidence map›Paper›PMID 33121959›Full record

ArticleOphthalmology2021

Evaluation of a Deep Learning-Derived Quantitative Retinopathy of Prematurity Severity Scale.

J Peter Campbell, Sang Jin Kim, James M Brown, Susan Ostmo, R V Paul Chan, Jayashree Kalpathy-Cramer, Michael F Chiang, of the Imaging and Informatics in Retinopathy of Prematurity Consortium

Abstract readComparative StudyMulticenter Study
In one paragraph

Article in Ophthalmology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 42 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
42citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

42 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Trial
  4. Article
  5. Article
  6. Current Trends in AI and Eye Disease Diagnostics.Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians (Optometrists) · 2026
    Review
  7. Quantifying Sagittal Craniosynostosis Severity: A Machine Learning Approach With CranioRate.The Cleft palate-craniofacial journal : official publication of the American Cleft Palate-Craniofacial Association · 2026
    Article
  8. [Research progress in artificial intelligence for the diagnosis and management of diseases in preterm infants].Zhongguo dang dai er ke za zhi = Chinese journal of contemporary pediatrics · 2026
    Review
  9. Article
  10. External validation of an artificial intelligence-based model for retinopathy of prematurity screening using Phoenix ICON retinal images.Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus · 2025
    Article
  11. Review
  12. Article
  13. Article
  14. Review
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Artificial intelligence in retinal disease: clinical application, challenges, and future directions.Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie · 2023
    Review
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

8 authors.

J Peter CampbellDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
Sang Jin KimDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
James M BrownSchool of Computer Science, University of Lincoln, Lincoln, United Kingdom.
Susan OstmoDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
R V Paul ChanDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Jayashree Kalpathy-CramerAthinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts; Massachusetts General Hospital and Brigham and Women's Hospital Center for Clinical Data Science, Boston, Massachusetts.
Michael F ChiangDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon. Electronic address: michael.chiang@nih.gov.
of the Imaging and Informatics in Retinopathy of Prematurity Consortium

Funding

Proteomics CoreP30EY010572 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI John Peter Campbell · 1995 to 2026
$19.4M
Clinical and Genetic Analysis of Retinopathy of PrematurityR01EY019474 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI CAMPBELL, JOHN PETER · 2010 to 2020
$7.0M
Mentored Vision Clinician-Scientist Program at OHSUK12EY027720 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI David Huang · 2018 to 2026
$2.2M
Clinical and genetic analysis of retinopathy of prematurityR01HD107493 · NICHD · OREGON HEALTH & SCIENCE UNIVERSITY · PI CAMPBELL, JOHN PETER · 2021 to 2023
$1.8M
Robust AI to develop risk models in retinopathy of prematurity using deep learningR21EY031883 · NEI · MASSACHUSETTS GENERAL HOSPITAL · PI KALPATHY-CRAMER, JAYASHREE, RUBIN, DANIEL L · 2020 to 2021
$472k
NEI NIH HHS K12 EY027720NEI NIH HHS P30 EY010572NEI NIH HHS R01 EY019474NEI NIH HHS R21 EY031883NICHD NIH HHS R01 HD107493
6 · The paper itself

Abstract

purposeTo evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale.

designAnalysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity.

participantsImages were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9.

methodsA quantitative vascular severity score (1-9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement.

main outcome measuresDistribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (<3 clock hours, 3-6 clock hours, and >6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale.

resultsFor deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale.

conclusionsA vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis.

Indexed as

AlgorithmsDeep LearningFollow-Up StudiesGestational AgeHumansInfant, NewbornOphthalmoscopyRetinal VesselsRetinopathy of PrematurityRetrospective StudiesSeverity of Illness Index

Identifiers

PMID33121959
PMCPMC8076329

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.