Evidence map›Paper›PMID 37113471›Full record

ArticleOphthalmology science2023

Characteristics of a Large, Labeled Data Set for the Training of Artificial Intelligence for Glaucoma Screening with Fundus Photographs.

Hans G Lemij, Coen de Vente, Clara I Sánchez, Koen A Vermeer

Registry-linked trialOpen access · goldAbstract read
In one paragraph

Article in Ophthalmology science, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT07668193 (GlaukomAI), which is not on this map. Cited by 9 papers.

0numbers the graph read from it
0cells of the map it votes in
9citing papers in PubMed
7.3field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT07668193 nanot yet recruitingnot on this mapstarted 2026, after this paper: background citation

GlaukomAI: Clinical Validation of an Artificial Intelligence-Based System for Early Glaucoma Screening and Diagnosis - A Case-Control Study and Referral Accuracy Assessment

TypeinterventionalSponsorFondazione G.B. Bietti, IRCCSRan2026 to 2028Enrolled1,200ConditionsGlaucomaArmsGlaukomAI (Sens-vue GlaukomAI)
3 · Its place in the literature

Who cites it

9 citing papers in PubMed, 32 citations in OpenAlex.

  1. Article
  2. JustRAIGS: Justified Referral in AI Glaucoma Screening Challenge.IEEE transactions on medical imaging · 2026
    Article
  3. Benchmarking Vision Encoders for Image Classification in Ophthalmology.Computational and structural biotechnology journal · 2026
    Article
  4. Article
  5. Article
  6. Review
  7. Article
  8. Review
  9. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors at 2 institutions in 2 countries.

Hans G LemijThe Rotterdam Eye Hospital, Rotterdam, the Netherlands.
Coen de VenteQuantitative Healthcare Analysis (QurAI) Group, Informatics Institute, University of Amsterdam, Amsterdam, the Netherlands.
Clara I SánchezQuantitative Healthcare Analysis (QurAI) Group, Informatics Institute, University of Amsterdam, Amsterdam, the Netherlands.
Koen A VermeerThe Rotterdam Ophthalmic Institute, Rotterdam Eye Hospital, Rotterdam, the Netherlands.
Quantitative BioSciences · USRotterdam Eye Hospital · NL

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Purpose: Significant visual impairment due to glaucoma is largely caused by the disease being detected too late. Objective: To build a labeled data set for training artificial intelligence (AI) algorithms for glaucoma screening by fundus photography, to assess the accuracy of the graders, and to characterize the features of all eyes with referable glaucoma (RG). Design: Cross-sectional study. Subjects: Color fundus photographs (CFPs) of 113 893 eyes of 60 357 individuals were obtained from EyePACS, California, United States, from a population screening program for diabetic retinopathy. Methods: Carefully selected graders (ophthalmologists and optometrists) graded the images. To qualify, they had to pass the European Optic Disc Assessment Trial optic disc assessment with ≥ 85% accuracy and 92% specificity. Of 90 candidates, 30 passed. Each image of the EyePACS set was then scored by varying random pairs of graders as "RG," "no referable glaucoma (NRG)," or "ungradable (UG)." In case of disagreement, a glaucoma specialist made the final grading. Referable glaucoma was scored if visual field damage was expected. In case of RG, graders were instructed to mark up to 10 relevant glaucomatous features. Main Outcome Measures: Qualitative features in eyes with RG. Results: The performance of each grader was monitored; if the sensitivity and specificity dropped below 80% and 95%, respectively (the final grade served as reference), they exited the study and their gradings were redone by other graders. In all, 20 graders qualified; their mean sensitivity and specificity (standard deviation [SD]) were 85.6% (5.7) and 96.1% (2.8), respectively. The 2 graders agreed in 92.45% of the images (Gwet's AC2, expressing the inter-rater reliability, was 0.917). Of all gradings, the sensitivity and specificity (95% confidence interval) were 86.0 (85.2-86.7)% and 96.4 (96.3-96.5)%, respectively. Of all gradable eyes ( Conclusions: A large data set of CFPs was put together of sufficient quality to develop AI screening solutions for glaucoma. The most common features of RG were the appearance of the NRR inferiorly and superiorly. Disc hemorrhages were a rare feature of RG. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

Indexed as

Artificial intelligenceClinical featurescolor fundus photographsglaucoma screeninglabeled data set

Identifiers

PMID37113471
PMCPMC10127130
OpenAlexW4327748037

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.