Evidence map›Paper›PMID 27566853›Full record

ArticleOphthalmology2016

Plus Disease in Retinopathy of Prematurity: Improving Diagnosis by Ranking Disease Severity and Using Quantitative Image Analysis.

Jayashree Kalpathy-Cramer, J Peter Campbell, Deniz Erdogmus, Peng Tian, Dharanish Kedarisetti, Chace Moleta, James D Reynolds, Kelly Hutcheson, Michael J Shapiro, Michael X Repka and 10 more

Open access · greenAbstract read
In one paragraph

Article in Ophthalmology, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 50 papers.

0numbers the graph read from it
0cells of the map it votes in
50citing papers in PubMed
6.5field-weighted citation impact, top 3% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

50 citing papers in PubMed, 82 citations in OpenAlex.

  1. Trial
  2. Trial
  3. Article
  4. Review
  5. Article
  6. P-Score Variability as a Function of Retinopathy of Prematurity Severity.Investigative ophthalmology & visual science · 2026
    Article
  7. Review
  8. Applications of generative adversarial networks in the diagnosis, prognosis, and treatment of ophthalmic diseases.Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie · 2025
    Review
  9. Article
  10. Article
  11. Article
  12. Review
  13. Article
  14. Article
  15. Complete blood count parameters as biomarkers of retinopathy of prematurity: a Portuguese multicenter study.Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie · 2023
    Observational
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

20 authors at 10 institutions in 2 countries.

Jayashree Kalpathy-CramerAthinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts.
J Peter CampbellDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
Deniz ErdogmusCognitive Systems Laboratory, Northeastern University, Boston, Massachusetts.
Peng TianCognitive Systems Laboratory, Northeastern University, Boston, Massachusetts.
Dharanish KedarisettiCognitive Systems Laboratory, Northeastern University, Boston, Massachusetts.
Chace MoletaDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
James D ReynoldsDepartment of Ophthalmology, Ross Eye Institute, State University of New York at Buffalo, Buffalo, New York.
Kelly HutchesonDepartment of Ophthalmology, Sidra Medical & Research Center, Doha, Qatar.
Michael J ShapiroRetina Consultants, Chicago, Illinois.
Michael X RepkaWilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland.
Philip FerroneLong Island Vitreoretinal Consultants, Great Neck, New York.
Kimberly DrenserAssociated Retinal Consultants, Oakland University, Royal Oak, Michigan.
Jason HorowitzDepartment of Ophthalmology, Columbia University, New York, New York.
Kemal SonmezDepartment of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon.
Ryan SwanDepartment of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon.
Susan OstmoDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
Karyn E JonasDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
R V Paul ChanDepartment of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
Michael F ChiangDepartment of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon. Electronic address: chiangm@ohsu.edu.
Imaging and Informatics in Retinopathy of Prematurity Research Consortium
Oregon Health & Science University · USNortheastern University · USUniversity of Illinois Chicago · USColumbia University · USMassachusetts General Hospital · USOakland University · USOphthalmic Consultants of Long Island · USJohns Hopkins University · USSidra Medical and Research Center · QAUniversity at Buffalo, State University of New York · US

Funding

Proteomics CoreP30EY010572 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI TED S ACOTT · 1995 to 2026
$19.4M
Clinical and Genetic Analysis of Retinopathy of PrematurityR01EY019474 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI CAMPBELL, JOHN PETER · 2010 to 2020
$7.0M
Translational Vision Science Research at Oregon Health & Science UniversityT32EY023211 · NEI · OREGON HEALTH & SCIENCE UNIVERSITY · PI Yali Jia, Kate E Keller · 2013 to 2026
$2.4M
Oregon Clinical and Translational Research InstituteTL1TR000129 · NCATS · OREGON HEALTH & SCIENCE UNIVERSITY · PI ELLISON, DAVID H · 2012 to 2015
$613k
Automated retinopathy of prematurity classification using machine learningR21EY022387 · NEI · MASSACHUSETTS GENERAL HOSPITAL · PI CHIANG, MICHAEL F., KALPATHY-CRAMER, JAYASHREE · 2013 to 2014
$482k
NCATS NIH HHS TL1 TR000129NEI NIH HHS P30 EY010572NEI NIH HHS R01 EY019474NEI NIH HHS R21 EY022387NEI NIH HHS T32 EY023211
6 · The paper itself

Abstract

purposeTo determine expert agreement on relative retinopathy of prematurity (ROP) disease severity and whether computer-based image analysis can model relative disease severity, and to propose consideration of a more continuous severity score for ROP.

designWe developed 2 databases of clinical images of varying disease severity (100 images and 34 images) as part of the Imaging and Informatics in ROP (i-ROP) cohort study and recruited expert physician, nonexpert physician, and nonphysician graders to classify and perform pairwise comparisons on both databases.

participantsSix participating expert ROP clinician-scientists, each with a minimum of 10 years of clinical ROP experience and 5 ROP publications, and 5 image graders (3 physicians and 2 nonphysician graders) who analyzed images that were obtained during routine ROP screening in neonatal intensive care units.

methodsImages in both databases were ranked by average disease classification (classification ranking), by pairwise comparison using the Elo rating method (comparison ranking), and by correlation with the i-ROP computer-based image analysis system.

main outcome measuresInterexpert agreement (weighted κ statistic) compared with the correlation coefficient (CC) between experts on pairwise comparisons and correlation between expert rankings and computer-based image analysis modeling.

resultsThere was variable interexpert agreement on diagnostic classification of disease (plus, preplus, or normal) among the 6 experts (mean weighted κ, 0.27; range, 0.06-0.63), but good correlation between experts on comparison ranking of disease severity (mean CC, 0.84; range, 0.74-0.93) on the set of 34 images. Comparison ranking provided a severity ranking that was in good agreement with ranking obtained by classification ranking (CC, 0.92). Comparison ranking on the larger dataset by both expert and nonexpert graders demonstrated good correlation (mean CC, 0.97; range, 0.95-0.98). The i-ROP system was able to model this continuous severity with good correlation (CC, 0.86).

conclusionsExperts diagnose plus disease on a continuum, with poor absolute agreement on classification but good relative agreement on disease severity. These results suggest that the use of pairwise rankings and a continuous severity score, such as that provided by the i-ROP system, may improve agreement on disease severity in the future.

Indexed as

Clinical CompetenceDiagnostic Techniques, OphthalmologicalHumansImage Processing, Computer-AssistedInfant, NewbornReproducibility of ResultsRetinaRetinopathy of PrematurityROC CurveSeverity of Illness Index

Identifiers

PMID27566853
PMCPMC5077696
OpenAlexW2507467374

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.