Evidence map›Paper›PMID 39812775›Full record

ArticleNeuroradiology2025

Evaluating a clinically available artificial intelligence model for intracranial aneurysm detection: a multi-reader study and algorithmic audit.

Bin Hu, Haitao He, Zhao Shi, Li Wang, Quanhui Liu, Zhiyuan Sun, Longjiang Zhang

Abstract read
PubMed Publisher
In one paragraph

Article in Neuroradiology, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 2 papers.

0numbers the graph read from it
0cells of the map it votes in
2citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

2 citing papers in PubMed.

  1. Article
  2. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Bin Hu *Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, Jiangsu, China.
Haitao He *Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, Jiangsu, China.
Zhao ShiDepartment of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, Jiangsu, China.
Li WangDepartment of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, Jiangsu, China.
Quanhui LiuDepartment of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, Jiangsu, China.
Zhiyuan SunDepartment of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, Jiangsu, China. sunlight1975@126.com.
Longjiang ZhangDepartment of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, 210002, Jiangsu, China. kevinzhlj@163.com.ORCID http://orcid.org/0000-0002-6664-7224

Funding

China National Funds for Distinguished Young Scientists 82102155Key Programme 82230068
6 · The paper itself

Abstract

purposeWe aimed to validate a clinically available artificial intelligence (AI) model to assist general radiologists in the detection of intracranial aneurysm (IA) in a multi-reader multi-case (MRMC) study, and to explore its performance in routine clinical settings.

methodsTwo distinct cohorts of head CT angiography (CTA) data were assembled to validate an AI model. Cohort 1, comprising gold-standard consecutive CTA cases, was used in an MRMC study involving six board-certified general radiologists. Cohort 2, representing clinical CTA cases, was used to simulate a routine clinical setting. Following these evaluations, an algorithmic audit was conducted to identify any unusual or unexpected behaviors exhibited by the model.

resultsCohort 1 consisted of 131 CTA cases, while Cohort 2 included 515 CTA cases. In the MRMC study, the AI-assisted strategy demonstrated a significant improvement in aneurysm diagnostic performance, with the area under the receiver operating characteristic curve increasing from 0.815 (95%CI: 0.754-0.875) to 0.875 (95%CI: 0.831-0.921; p = 0.008). In the AI-based first-reader study, 60.4% of the CTA cases were identified as negative by the AI, with a high negative predictive value of 0.994 (95%CI: 0.977-0.999). The algorithmic audit highlighted two issues for improvement: the accurate detection of tiny aneurysms and the effective exclusion of false-positive lesions.

conclusionThis study highlights the clinical utility of a high-performance AI model in detecting IAs, significantly improving general radiologists' diagnostic performance with the potential to reduce their workload in routine clinical practice. The algorithmic audit offers insights to guide the development and validation of future AI models.

Indexed as

AlgorithmsArtificial IntelligenceCerebral AngiographyComputed Tomography AngiographyIntracranial AneurysmAdultAgedCohort StudiesFemaleHumansMaleMiddle AgedRadiographic Image Interpretation, Computer-AssistedSensitivity and SpecificityArtificial intelligenceCommercial productDeep learningIntracranial aneurysmMulti-reader multi-case study

Identifiers

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.