ArticleDiabetes care2021
Multicenter, Head-to-Head, Real-World Validation Study of Seven Automated Artificial Intelligence Diabetic Retinopathy Screening Systems.
Article in Diabetes care, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 107 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
107 citing papers in PubMed, 5 syntheses or guidelines pooled it, 161 citations in OpenAlex.
- Transformative applications of oculomics-based AI approaches in the management of systemic diseases: A systematic review.Journal of advanced research · 2025Pooled it
- Artificial Intelligence improves follow-up appointment uptake for diabetic retinal assessment: a systematic review and meta-analysis.Eye (London, England) · 2025Pooled it
- Performance and limitation of machine learning algorithms for diabetic retinopathy screening and its application in health management: a meta-analysis.Biomedical engineering online · 2025Pooled it
- Predicting systemic diseases in fundus images: systematic review of setting, reporting, bias, and models' clinical availability in deep learning studies.Eye (London, England) · 2024Pooled it
- Diagnostic accuracy and potential covariates of artificial intelligence for diagnosing orthopedic fractures: a systematic literature review and meta-analysis.European radiology · 2022Pooled it
- Pivotal Evaluation of an Artificial Intelligence System for Autonomous Detection of Referrable and Vision-Threatening Diabetic Retinopathy.JAMA network open · 2021Trial
- Clinical Effectiveness of AI-Based Cardiac Arrest Prediction System for General Ward Patients (VITAL-MD): Protocol for a Multicenter Pragmatic Stepped Wedge Cluster Randomized Controlled Trial.JMIR research protocols · 2026Article
- Artificial intelligence in ophthalmology: From diagnostic accuracy to clinical application.World journal of methodology · 2026Review
- Multimodal AI in retinal disease: from images to decisions and real-world endpoints.International ophthalmology · 2026Article
- Early Detection of Glaucoma and Diabetic Retinopathy in Low-Resource Settings: Barriers and Solutions.Journal of clinical medicine · 2026Review
- Review
- Prospective Data Curation Enables High-Performance Artificial Intelligence for Diabetic Retinopathy Screening in a Resource-Limited Setting.Ophthalmology science · 2026Article
- Implementation of an AI-Based Clinical Decision Support System Predicting In-Hospital Cardiac Arrest in General Wards: A Multicenter Staggered-Implementation Study in Secondary Hospitals in Korea.Diagnostics (Basel, Switzerland) · 2026Article
- Impact of artificial intelligence on the availability, accessibility, acceptability and quality of ophthalmic disease screening services: a scoping review.The British journal of ophthalmology · 2026Article
- Rethinking scale in ophthalmic artificial intelligence: from bigger models to smarter clinical reasoning.NPJ digital medicine · 2026Review
- Real-World Performance of Artificial Intelligence in Diabetic Retinopathy Screening: A Systematic Review.Cureus · 2026Review
- Customizing AI-based screening with real-world data: Practical insights from diabetic retinopathy.Acta ophthalmologica · 2026Observational
- Artificial intelligence in retinal care: transforming the doctor-patient partnership.International journal of retina and vitreous · 2026Review
- BigEye: a clinically interpretable deep learning framework for diabetic retinopathy detection and stage prediction.Scientific reports · 2026Article
- Diagnostic Accuracy and Real-Life Advantages of the MONA.health Artificial Intelligence Software in Screening for Diabetic Retinopathy and Maculopathy.Diagnostics (Basel, Switzerland) · 2026Article
47 more citing papers are in PubMed but not listed here.
Corrections and comments
- Commented on by
- Commented on by
Authors and funding
12 authors at 4 institutions in 1 country.
Funding
Abstract
objectiveWith rising global prevalence of diabetic retinopathy (DR), automated DR screening is needed for primary care settings. Two automated artificial intelligence (AI)-based DR screening algorithms have U.S. Food and Drug Administration (FDA) approval. Several others are under consideration while in clinical use in other countries, but their real-world performance has not been evaluated systematically. We compared the performance of seven automated AI-based DR screening algorithms (including one FDA-approved algorithm) against human graders when analyzing real-world retinal imaging data. RESEARCH DESIGN AND
methodsThis was a multicenter, noninterventional device validation study evaluating a total of 311,604 retinal images from 23,724 veterans who presented for teleretinal DR screening at the Veterans Affairs (VA) Puget Sound Health Care System (HCS) or Atlanta VA HCS from 2006 to 2018. Five companies provided seven algorithms, including one with FDA approval, that independently analyzed all scans, regardless of image quality. The sensitivity/specificity of each algorithm when classifying images as referable DR or not were compared with original VA teleretinal grades and a regraded arbitrated data set. Value per encounter was estimated.
resultsAlthough high negative predictive values (82.72-93.69%) were observed, sensitivities varied widely (50.98-85.90%). Most algorithms performed no better than humans against the arbitrated data set, but two achieved higher sensitivities, and one yielded comparable sensitivity (80.47%,
conclusionsThe DR screening algorithms showed significant performance differences. These results argue for rigorous testing of all such algorithms on real-world data before clinical implementation.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.