Evidence map›Paper›PMID 36531570›Full record

ArticleOphthalmology science2022

Implementation of a Large-Scale Image Curation Workflow Using Deep Learning Framework.

Amitha Domalpally, Robert Slater, Nancy Barrett, Rick Voland, Rohit Balaji, Jennifer Heathcote, Roomasa Channa, Barbara Blodi

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Ophthalmology science, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

8 authors.

Amitha DomalpallyA-EYE Unit, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.
Robert SlaterA-EYE Unit, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.
Nancy BarrettWisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.
Rick VolandWisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.
Rohit BalajiWisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.
Jennifer HeathcoteWisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.
Roomasa ChannaA-EYE Unit, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.
Barbara BlodiWisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Purpose: The curation of images using human resources is time intensive but an essential step for developing artificial intelligence (AI) algorithms. Our goal was to develop and implement an AI algorithm for image curation in a high-volume setting. We also explored AI tools that will assist in deploying a tiered approach, in which the AI model labels images and flags potential mislabels for human review. Design: Implementation of an AI algorithm. Participants: Seven-field stereoscopic images from multiple clinical trials. Methods: The 7-field stereoscopic image protocol includes 7 pairs of images from various parts of the central retina along with images of the anterior part of the eye. All images were labeled for field number by reading center graders. The model output included classification of the retinal images into 8 field numbers. Probability scores (0-1) were generated to identify misclassified images, with 1 indicating a high probability of a correct label. Main Outcome Measures: Agreement of AI prediction with grader classification of field number and the use of probability scores to identify mislabeled images. Results: The AI model was trained and validated on 17 529 images and tested on 3004 images. The pooled agreement of field numbers between grader classification and the AI model was 88.3% (kappa, 0.87). The pooled mean probability score was 0.97 (standard deviation [SD], 0.08) for images for which the graders agreed with the AI-generated labels and 0.77 (SD, 0.19) for images for which the graders disagreed with the AI-generated labels ( Conclusions: The implementation of AI algorithms requires measures in addition to model validation. Tools to flag potential errors in the labels generated by AI models will reduce inaccuracies, increase trust in the system, and provide data for continuous model development.

Indexed as

AI, artificial intelligenceArtificial intelligenceDeep learningFundus photographImage curationMachine learningMetadataRetinal imageStandardization

Identifiers

PMID36531570
PMCPMC9754974

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.