Evidence map›Paper›PMID 35659941›Full record

ArticleOphthalmology. Retina2022

Variability in Plus Disease Diagnosis using Single and Serial Images.

Emily D Cole, Shin Hae Park, Sang Jin Kim, Kai B Kang, Nita G Valikodath, Tala Al-Khaled, Samir N Patel, Karyn E Jonas, Susan Ostmo, Aaron Coyner and 10 more

Abstract read
In one paragraph

Article in Ophthalmology. Retina, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers.

0numbers the graph read from it
0cells of the map it votes in
9citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

9 citing papers in PubMed.

  1. Article
  2. Impact of image enhancement on grading agreement of plus disease features in retinopathy of prematurity.Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie · 2026
    Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Review
  9. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

20 authors.

Emily D ColeDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Shin Hae ParkDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois; Department of Ophthalmology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea.
Sang Jin KimDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea.
Kai B KangDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Nita G ValikodathDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Tala Al-KhaledDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Samir N PatelWills Eye Hospital, Philadelphia, Pennsylvania.
Karyn E JonasDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Susan OstmoDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
Aaron CoynerDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
Audina BerrocalBascom Palmer Eye Institute, University of Miami, Miami, Florida.
Kimberly A DrenserDepartment of Ophthalmology, Beaumont Eye Institute, Royal Oak, Michigan.
Aaron NagielStein Eye Institute, University of California Los Angeles, Los Angeles, California.
Jason D HorowitzDepartment of Ophthalmology, Columbia University, New York, New York.
Thomas C LeeRoski Eye Institute, Department of Ophthalmology, Keck School of Medicine of the University of Southern California, Los Angeles, California.
Jayashree Kalpathy-CramerHarvard Medical School, Boston, Massachusetts.
Michael F ChiangNational Eye Institute, National Institutes of Health, Bethesda, Maryland.
J Peter CampbellDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
R V Paul ChanDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois. Electronic address: rvpchan@uic.edu.
Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium

Funding

Proteomics CoreP30EY010572 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI TED S ACOTT · 1995 to 2026
$19.4M
Translational Core for Therapeutic and Diagnostic DevelopmentP30EY001792 · NEI · UNIVERSITY OF ILLINOIS AT CHICAGO · PI MCANANY, JAMES JASON · 1985 to 2025
$14.8M
Clinical and Genetic Analysis of Retinopathy of PrematurityR01EY019474 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI CAMPBELL, JOHN PETER · 2010 to 2020
$7.0M
Artificial Intelligence Assisted Panoramic Optical Coherence Tomography Angiography for Retinopathy of PrematurityR01HD107494 · NICHD · OREGON HEALTH & SCIENCE UNIVERSITY · PI John Peter Campbell, Yifan Jian · 2021 to 2026
$2.4M
Mentored Vision Clinician-Scientist Program at OHSUK12EY027720 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI David Huang · 2018 to 2026
$2.2M
Clinical and genetic analysis of retinopathy of prematurityR01HD107493 · NICHD · OREGON HEALTH & SCIENCE UNIVERSITY · PI CAMPBELL, JOHN PETER · 2021 to 2023
$1.8M
Artificial intelligence assisted panoramic Optical Coherence Tomography Angiography for Retinopathy of PrematurityR01EY031331 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI CAMPBELL, JOHN PETER, JIAN, YIFAN · 2020 to 2020
$342k
NEI NIH HHS K12 EY027720NEI NIH HHS P30 EY001792NEI NIH HHS P30 EY010572NEI NIH HHS R01 EY019474NEI NIH HHS R01 EY031331NICHD NIH HHS R01 HD107493NICHD NIH HHS R01 HD107494
6 · The paper itself

Abstract

purposeTo assess changes in retinopathy of prematurity (ROP) diagnosis in single and serial retinal images.

designCohort study.

participantsCases of ROP recruited from the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) consortium evaluated by 7 graders.

methodsSeven ophthalmologists reviewed both single and 3 consecutive serial retinal images from 15 cases with ROP, and severity was assigned as plus, preplus, or none. Imaging data were acquired during routine ROP screening from 2011 to 2015, and a reference standard diagnosis was established for each image. A secondary analysis was performed using the i-ROP deep learning system to assign a vascular severity score (VSS) to each image, ranging from 1 to 9, with 9 being the most severe disease. This score has been previously demonstrated to correlate with the International Classification of ROP. Mean plus disease severity was calculated by averaging 14 labels per image in serial and single images to decrease noise.

main outcome measuresGrading severity of ROP as defined by plus, preplus, or no ROP.

resultsAssessment of serial retinal images changed the grading severity for > 50% of the graders, although there was wide variability. Cohen's kappa ranged from 0.29 to 1.0, which showed a wide range of agreement from slight to perfect by each grader. Changes in the grading of serial retinal images were noted more commonly in cases of preplus disease. The mean severity in cases with a diagnosis of plus disease and no disease did not change between single and serial images. The ROP VSS demonstrated good correlation with the range of expert classifications of plus disease and overall agreement with the mode class (P = 0.001). The VSS correlated with mean plus disease severity by expert diagnosis (correlation coefficient, 0.89). The more aggressive graders tended to be influenced by serial images to increase the severity of their grading. The VSS also demonstrated agreement with disease progression across serial images, which progressed to preplus and plus disease.

conclusionsClinicians demonstrated variability in ROP diagnosis when presented with both single and serial images. The use of deep learning as a quantitative assessment of plus disease has the potential to standardize ROP diagnosis and treatment.

Indexed as

Retinopathy of PrematurityTelemedicineCohort StudiesDiagnostic ImagingHumansInfant, NewbornReproducibility of ResultsRetinopathy of prematurityROP imagingROP progressionROP screeningtelemedicine

Identifiers

PMID35659941
PMCPMC10278151

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.