Evidence map›Paper›PMID 33629545›Full record

ReviewKorean journal of radiology2021

Key Principles of Clinical Validation, Device Approval, and Insurance Coverage Decisions of Artificial Intelligence.

Seong Ho Park, Jaesoon Choi, Jeong Sik Byeon

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in Korean journal of radiology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 66 papers, 3 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
66citing papers in PubMed, 3 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

66 citing papers in PubMed, 3 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Article
  5. Article
  6. Review
  7. Article
  8. Article
  9. Review
  10. Review
  11. Review
  12. Artificial intelligence in ophthalmology clinical trials: a narrative review.Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie · 2026
    Review
  13. Article
  14. Review
  15. Article
  16. Article
  17. Review
  18. Review
  19. Review
  20. Article

6 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Seong Ho ParkDepartment of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea. seongho@amc.seoul.kr.ORCID 0000-0002-1257-8315
Jaesoon ChoiDepartment of Biomedical Engineering, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea.ORCID 0000-0002-6817-618X
Jeong Sik ByeonDepartment of Gastroenterology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea.ORCID 0000-0002-9793-6379

Funding

Korea Health Industry Development Institute HI17C2410
6 · The paper itself

Abstract

Artificial intelligence (AI) will likely affect various fields of medicine. This article aims to explain the fundamental principles of clinical validation, device approval, and insurance coverage decisions of AI algorithms for medical diagnosis and prediction. Discrimination accuracy of AI algorithms is often evaluated with the Dice similarity coefficient, sensitivity, specificity, and traditional or free-response receiver operating characteristic curves. Calibration accuracy should also be assessed, especially for algorithms that provide probabilities to users. As current AI algorithms have limited generalizability to real-world practice, clinical validation of AI should put it to proper external testing and assisting roles. External testing could adopt diagnostic case-control or diagnostic cohort designs. A diagnostic case-control study evaluates the technical validity/accuracy of AI while the latter tests the clinical validity/accuracy of AI in samples representing target patients in real-world clinical scenarios. Ultimate clinical validation of AI requires evaluations of its impact on patient outcomes, referred to as clinical utility, and for which randomized clinical trials are ideal. Device approval of AI is typically granted with proof of technical validity/accuracy and thus does not intend to directly indicate if AI is beneficial for patient care or if it improves patient outcomes. Neither can it categorically address the issue of limited generalizability of AI. After achieving device approval, it is up to medical professionals to determine if the approved AI algorithms are beneficial for real-world patient care. Insurance coverage decisions generally require a demonstration of clinical utility that the use of AI has improved patient outcomes.

Indexed as

Artificial IntelligenceDevice ApprovalInsurance CoverageArea Under CurveCase-Control StudiesDecision MakingDelivery of Health CareHumansLung NeoplasmsROC CurveArtificial intelligenceDevice approvalInsurance coverageSoftware validation

Identifiers

PMID33629545
PMCPMC7909857

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.