ArticleBMJ health & care informatics2021
Development and validation pathways of artificial intelligence tools evaluated in randomised clinical trials.
Article in BMJ health & care informatics, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 21 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
21 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Adherence of randomised controlled trials using artificial intelligence in ophthalmology to CONSORT-AI guidelines: a systematic review and critical appraisal.BMJ health & care informatics · 2023Pooled it
- Clinical Evidence and FDA Recalls of Artificial Intelligence-Enabled Medical Devices.JAMA network open · 2026Article
- Artificial intelligence 12-lead electrocardiography to determine atrial fibrillation risk among UK Biobank participants with predisposing conditions.European heart journal. Digital health · 2026Article
- Myocardial Revascularization in 2025: A Clinical Perspective on the Evolution of Technologies, Strategic Decision-Making, and Future Horizons.Reviews in cardiovascular medicine · 2026Review
- Validation of the Use of a Large Language Model for Detecting Sentiment in Student Course Evaluation.Family medicine · 2026Article
- Assessing User Experience and Satisfaction with a Mobile Application for Drug Dosage Calculation-A Pilot Study.Dentistry journal · 2026Article
- Artificial intelligence in acute and critical care: current challenges and strategic solutions.Frontiers in public health · 2026Review
- The expanding role of artificial intelligence in personalised medicine: from innovation to individualized care.Frontiers in medicine · 2026Review
- Tooth-level detection and mapping of dental pathologies on panoramic radiographs using YOLOv11 and RT-DETR.MethodsX · 2025Article
- Impact of Clinical Trial Virtualization on Recruitment in Underserved Communities for a Type 2 Diabetes mHealth Intervention.medRxiv : the preprint server for health sciences · 2025Article
- Opportunities for Artificial Intelligence in Oncology: From the Lens of Clinicians and Patients.JCO oncology practice · 2025Review
- Economic Evaluations and Equity in the Use of Artificial Intelligence in Imaging Examinations for Medical Diagnosis in People With Dermatological, Neurological, and Pulmonary Diseases: Systematic Review.Interactive journal of medical research · 2025Review
- Generalizability of FDA-Approved AI-Enabled Medical Devices for Clinical Use.JAMA network open · 2025Article
- Can AI Be Useful in the Early Detection of Pancreatic Cancer in Patients with New-Onset Diabetes?Biomedicines · 2025Review
- Detection of severe aortic stenosis by clinicians versus artificial intelligence: A retrospective clinical cohort study.American heart journal plus : cardiology research and practice · 2024Article
- Automating Dental Condition Detection on Panoramic Radiographs: Challenges, Pitfalls, and Opportunities.Diagnostics (Basel, Switzerland) · 2024Article
- GPT-Driven Radiology Report Generation with Fine-Tuned Llama 3.Bioengineering (Basel, Switzerland) · 2024Article
- Artificial Intelligence in Head and Neck Cancer: Innovations, Applications, and Future Directions.Current oncology (Toronto, Ont.) · 2024Review
- Hypertrophic cardiomyopathy detection with artificial intelligence electrocardiography in international cohorts: an external validation study.European heart journal. Digital health · 2024Article
- Automatic assessment of atherosclerotic plaque features by intracoronary imaging: a scoping review.Frontiers in cardiovascular medicine · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveGiven the complexities of testing the translational capability of new artificial intelligence (AI) tools, we aimed to map the pathways of training/validation/testing in development process and external validation of AI tools evaluated in dedicated randomised controlled trials (AI-RCTs).
methodsWe searched for peer-reviewed protocols and completed AI-RCTs evaluating the clinical effectiveness of AI tools and identified development and validation studies of AI tools. We collected detailed information, and evaluated patterns of development and external validation of AI tools.
resultsWe found 23 AI-RCTs evaluating the clinical impact of 18 unique AI tools (2009-2021). Standard-of-care interventions were used in the control arms in all but one AI-RCT. Investigators did not provide access to the software code of the AI tool in any of the studies. Considering the primary outcome, the results were in favour of the AI intervention in 82% of the completed AI-RCTs (14 out of 17). We identified significant variation in the patterns of development, external validation and clinical evaluation approaches among different AI tools. A published development study was found only for 10 of the 18 AI tools. Median time from the publication of a development study to the respective AI-RCT was 1.4 years (IQR 0.2-2.2).
conclusionsWe found significant variation in the patterns of development and validation for AI tools before their evaluation in dedicated AI-RCTs. Published peer-reviewed protocols and completed AI-RCTs were also heterogeneous in design and reporting. Upcoming guidelines providing guidance for the development and clinical translation process aim to improve these aspects.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.