Evidence map›Paper›PMID 27591053›Full record

ArticleOphthalmology2016

Plus Disease in Retinopathy of Prematurity: A Continuous Spectrum of Vascular Abnormality as a Basis of Diagnostic Variability.

J Peter Campbell, Jayashree Kalpathy-Cramer, Deniz Erdogmus, Peng Tian, Dharanish Kedarisetti, Chace Moleta, James D Reynolds, Kelly Hutcheson, Michael J Shapiro, Michael X Repka and 10 more

Registry-linked trialOpen access · greenAbstract read
In one paragraph

Article in Ophthalmology, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT04995341 (Analyzing Retinal Microanatomy in Retinopathy of Prematurity to Improve Care 2 and School Age Follow on Study), which is not on this map. Cited by 56 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
56citing papers in PubMed, 1 pooled it
7.4field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT04995341 narecruitingnot on this mapstarted 2021, after this paper: background citation

Analyzing Retinal Microanatomy in Retinopathy of Prematurity to Improve Care 2 and School Age Follow on Study (BabySTEPS2)

TypeinterventionalSponsorDuke UniversityRan2021 to 2027Enrolled236ConditionsRetinopathy of PrematurityArmsInvestigational ultracompact OCT and OCTA system, retinal photographs
3 · Its place in the literature

Who cites it

56 citing papers in PubMed, 1 synthesis or guideline pooled it, 86 citations in OpenAlex.

  1. Guideline
  2. Trial
  3. Article
  4. Impact of image enhancement on grading agreement of plus disease features in retinopathy of prematurity.Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie · 2026
    Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. External validation of an artificial intelligence-based model for retinopathy of prematurity screening using Phoenix ICON retinal images.Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus · 2025
    Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Ocular blood flow in preterm neonates.Scientific reports · 2024
    Article
  17. Article
  18. Observational
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

20 authors at 10 institutions in 2 countries.

J Peter CampbellDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
Jayashree Kalpathy-CramerAthinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts.
Deniz ErdogmusCognitive Systems Laboratory, Northeastern University, Boston, Massachusetts.
Peng TianCognitive Systems Laboratory, Northeastern University, Boston, Massachusetts.
Dharanish KedarisettiCognitive Systems Laboratory, Northeastern University, Boston, Massachusetts.
Chace MoletaDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
James D ReynoldsDepartment of Ophthalmology, Ross Eye Institute, State University of New York at Buffalo, Buffalo, New York.
Kelly HutchesonDepartment of Ophthalmology, Sidra Medical & Research Center, Doha, Qatar.
Michael J ShapiroRetina Consultants, Chicago, Illinois.
Michael X RepkaWilmer Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland.
Philip FerroneLong Island Vitreoretinal Consultants, Great Neck, New York.
Kimberly DrenserAssociated Retinal Consultants, Oakland University, Royal Oak, Michigan.
Jason HorowitzDepartment of Ophthalmology, Columbia University, New York, New York.
Kemal SonmezDepartment of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon.
Ryan SwanDepartment of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon.
Susan OstmoDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
Karyn E JonasDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
R V Paul ChanDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Michael F ChiangDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon. Electronic address: chiangm@ohsu.edu.
Imaging and Informatics in Retinopathy of Prematurity Research Consortium
Oregon Health & Science University · USNortheastern University · USUniversity of Illinois Chicago · USColumbia University · USMassachusetts General Hospital · USJohns Hopkins University · USOakland University · USOphthalmic Consultants of Long Island · USSidra Medical and Research Center · QAUniversity at Buffalo, State University of New York · US

Funding

Proteomics CoreP30EY010572 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI TED S ACOTT · 1995 to 2026
$19.4M
Clinical and Genetic Analysis of Retinopathy of PrematurityR01EY019474 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI CAMPBELL, JOHN PETER · 2010 to 2020
$7.0M
Translational Vision Science Research at Oregon Health & Science UniversityT32EY023211 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI Yali Jia, Kate E Keller · 2013 to 2026
$2.4M
Oregon Clinical and Translational Research InstituteTL1TR000129 · NCATS · OREGON HEALTH & SCIENCE UNIVERSITY · PI ELLISON, DAVID H · 2012 to 2015
$613k
Automated retinopathy of prematurity classification using machine learningR21EY022387 · NEI · MASSACHUSETTS GENERAL HOSPITAL · PI CHIANG, MICHAEL F., KALPATHY-CRAMER, JAYASHREE · 2013 to 2014
$482k
NCATS NIH HHS TL1 TR000129NEI NIH HHS P30 EY010572NEI NIH HHS R01 EY019474NEI NIH HHS R21 EY022387NEI NIH HHS T32 EY023211
6 · The paper itself

Abstract

purposeTo identify patterns of interexpert discrepancy in plus disease diagnosis in retinopathy of prematurity (ROP).

designWe developed 2 datasets of clinical images as part of the Imaging and Informatics in ROP study and determined a consensus reference standard diagnosis (RSD) for each image based on 3 independent image graders and the clinical examination results. We recruited 8 expert ROP clinicians to classify these images and compared the distribution of classifications between experts and the RSD.

participantsEight participating experts with more than 10 years of clinical ROP experience and more than 5 peer-reviewed ROP publications who analyzed images obtained during routine ROP screening in neonatal intensive care units.

methodsExpert classification of images of plus disease in ROP.

main outcome measuresInterexpert agreement (weighted κ statistic) and agreement and bias on ordinal classification between experts (analysis of variance [ANOVA]) and the RSD (percent agreement).

resultsThere was variable interexpert agreement on diagnostic classifications between the 8 experts and the RSD (weighted κ, 0-0.75; mean, 0.30). The RSD agreement ranged from 80% to 94% for the dataset of 100 images and from 29% to 79% for the dataset of 34 images. However, when images were ranked in order of disease severity (by average expert classification), the pattern of expert classification revealed a consistent systematic bias for each expert consistent with unique cut points for the diagnosis of plus disease and preplus disease. The 2-way ANOVA model suggested a highly significant effect of both image and user on the average score (dataset A: P < 0.05 and adjusted R

conclusionsThere is wide variability in the classification of plus disease by ROP experts, which occurs because experts have different cut points for the amounts of vascular abnormality required for presence of plus and preplus disease. This has important implications for research, teaching, and patient care for ROP and suggests that a continuous ROP plus disease severity score may reflect more accurately the behavior of expert ROP clinicians and may better standardize classification in the future.

Indexed as

Diagnosis, DifferentialFemaleHumansInfant, NewbornMaleNeonatal ScreeningPhotographyReproducibility of ResultsRetinaRetinal VesselsRetinopathy of PrematurityROC Curve

Identifiers

PMID27591053
PMCPMC5077639
OpenAlexW2615077572

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.