Evidence map›Paper›PMID 37269414›Full record

ArticleNeuroradiology2023

Validation of a deep learning model for traumatic brain injury detection and NIRIS grading on non-contrast CT: a multi-reader study with promising results and opportunities for improvement.

Bin Jiang, Burak Berksu Ozkara, Sean Creeden, Guangming Zhu, Victoria Y Ding, Hui Chen, Bryan Lanzman, Dylan Wolman, Sara Shams, Austin Trinh and 5 more

Abstract read
PubMed Publisher
In one paragraph

Article in Neuroradiology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 4 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
4citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

4 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Review
  4. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors.

Bin Jiang *Department of Radiology, Neuroradiology Division, Stanford University, Stanford, CA, USA.
Burak Berksu Ozkara *Department of Neuroradiology, MD Anderson Cancer Center, Houston, TX, USA.ORCID http://orcid.org/0000-0002-8769-3342
Sean CreedenDeparment of Neuroradiology, University of Illinois College of Medicine Peoria, Peoria, IL, USA.ORCID http://orcid.org/0000-0002-5361-1732
Guangming ZhuDepartment of Neurology, The University of Arizona, Tucson, AZ, USA.ORCID http://orcid.org/0000-0001-5519-3135
Victoria Y DingDepartment of Medicine, Stanford University, Stanford, CA, USA.ORCID http://orcid.org/0000-0002-2049-9051
Hui ChenDepartment of Neuroradiology, MD Anderson Cancer Center, Houston, TX, USA.ORCID http://orcid.org/0000-0002-4479-0185
Bryan LanzmanDepartment of Radiology, Neuroradiology Division, Stanford University, Stanford, CA, USA.ORCID http://orcid.org/0000-0002-6580-8540
Dylan WolmanDepartment of Neuroimaging and Neurointervention, Stanford University, Stanford, CA, USA.ORCID http://orcid.org/0000-0002-5012-1690
Sara ShamsDepartment of Radiology, Neuroradiology Division, Stanford University, Stanford, CA, USA.
Austin TrinhDepartment of Neuroimaging and Neurointervention, Stanford University, Stanford, CA, USA.
Ying LiDepartment of Radiology, Neuroradiology Division, Stanford University, Stanford, CA, USA.ORCID http://orcid.org/0000-0003-1001-5591
Alexander KhalafDepartment of Neuroimaging and Neurointervention, Stanford University, Stanford, CA, USA.ORCID http://orcid.org/0000-0001-6366-6318
Jonathon J ParkerDevice-Based Neuroelectronics Laboratory, Mayo Clinic, Phoenix, AZ, USA.ORCID http://orcid.org/0000-0002-0531-8400
Casey H HalpernDepartment of Neurosurgery, University of Pennsylvania School of Medicine, Philadelphia, PA, USA.ORCID http://orcid.org/0000-0003-3907-216X
Max WintermarkDepartment of Neuroradiology, MD Anderson Cancer Center, Houston, TX, USA. max.wintermark@gmail.com.ORCID http://orcid.org/0000-0002-6726-3951

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeThis study aimed to assess and externally validate the performance of a deep learning (DL) model for the interpretation of non-contrast computed tomography (NCCT) scans of patients with suspicion of traumatic brain injury (TBI).

methodsThis retrospective and multi-reader study included patients with TBI suspicion who were transported to the emergency department and underwent NCCT scans. Eight reviewers, with varying levels of training and experience (two neuroradiology attendings, two neuroradiology fellows, two neuroradiology residents, one neurosurgery attending, and one neurosurgery resident), independently evaluated NCCT head scans. The same scans were evaluated using the version 5.0 of the DL model icobrain tbi. The establishment of the ground truth involved a thorough assessment of all accessible clinical and laboratory data, as well as follow-up imaging studies, including NCCT and magnetic resonance imaging, as a consensus amongst the study reviewers. The outcomes of interest included neuroimaging radiological interpretation system (NIRIS) scores, the presence of midline shift, mass effect, hemorrhagic lesions, hydrocephalus, and severe hydrocephalus, as well as measurements of midline shift and volumes of hemorrhagic lesions. Comparisons using weighted Cohen's kappa coefficient were made. The McNemar test was used to compare the diagnostic performance. Bland-Altman plots were used to compare measurements.

resultsOne hundred patients were included, with the DL model successfully categorizing 77 scans. The median age for the total group was 48, with the omitted group having a median age of 44.5 and the included group having a median age of 48. The DL model demonstrated moderate agreement with the ground truth, trainees, and attendings. With the DL model's assistance, trainees' agreement with the ground truth improved. The DL model showed high specificity (0.88) and positive predictive value (0.96) in classifying NIRIS scores as 0-2 or 3-4. Trainees and attendings had the highest accuracy (0.95). The DL model's performance in classifying various TBI CT imaging common data elements was comparable to that of trainees and attendings. The average difference for the DL model in quantifying the volume of hemorrhagic lesions was 6.0 mL with a wide 95% confidence interval (CI) of - 68.32 to 80.22, and for midline shift, the average difference was 1.4 mm with a 95% CI of - 3.4 to 6.2.

conclusionWhile the DL model outperformed trainees in some aspects, attendings' assessments remained superior in most instances. Using the DL model as an assistive tool benefited trainees, improving their NIRIS score agreement with the ground truth. Although the DL model showed high potential in classifying some TBI CT imaging common data elements, further refinement and optimization are necessary to enhance its clinical utility.

Indexed as

Brain Injuries, TraumaticDeep LearningHydrocephalusHumansNeuroimagingRetrospective StudiesTomography, X-Ray ComputedComputed tomographyDeep learningNeuroimaging radiological interpretation systemTraumatic brain injuryValidation

Identifiers

PMID37269414

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.