Evidence map›Paper›PMID 38302827›Full record

ArticleEmergency radiology2024

An ASER AI/ML expert panel formative user research study for an interpretable interactive splenic AAST grading graphical user interface prototype.

Nathan Sarkar, Mitsuo Kumagai, Samantha Meyr, Sriya Pothapragada, Mathias Unberath, Guang Li, Sagheer Rauf Ahmed, Elana Beth Smith, Melissa Ann Davis, Garvit Devmohan Khatri and 5 more

Open access · greenAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Emergency radiology, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers.

0numbers the graph read from it
0cells of the map it votes in
5citing papers in PubMed
4.0field-weighted citation impact, top 6% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

5 citing papers in PubMed, 7 citations in OpenAlex.

  1. Article
  2. Review
  3. Human-centered explainability evaluation in clinical decision-making: a critical review of the literature.Journal of the American Medical Informatics Association : JAMIA · 2025
    Review
  4. Article
  5. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors at 7 institutions in 1 country.

Nathan SarkarUniversity of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA.
Mitsuo KumagaiUniversity of Maryland College Park, 4603 Calvert Rd, College Park, MD, 20740, USA.
Samantha MeyrUniversity of Maryland College Park, 4603 Calvert Rd, College Park, MD, 20740, USA.
Sriya PothapragadaUniversity of Maryland College Park, 4603 Calvert Rd, College Park, MD, 20740, USA.
Mathias UnberathJohns Hopkins University, 3400 N. Charles Street, Baltimore, MD, 21218, USA.
Guang LiUniversity of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA.
Sagheer Rauf AhmedUniversity of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA.
Elana Beth SmithUniversity of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA.
Melissa Ann DavisYale School of Medicine, 333 Cedar St, New Haven, CT, 06510, USA.
Garvit Devmohan KhatriUniversity of Colorado, 13001 E 17Th Pl, Aurora, CO, 80045, USA.
Anjali AgrawalTeleradiology Solutions, 22 Lianfair Road Unit 6, Ardmore, PA, 19003, USA.
Zachary Scott DelpropostoUniversity of Michigan Medical School, 1301 Catherine St, Ann Arbor, MI, 48109, USA.
Haomin ChenJohns Hopkins University, 3400 N. Charles Street, Baltimore, MD, 21218, USA.
Catalina Gómez CaballeroJohns Hopkins University, 3400 N. Charles Street, Baltimore, MD, 21218, USA.
David DreizinUniversity of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA. daviddreizin@gmail.com.ORCID http://orcid.org/0000-0002-0176-0912
University of Maryland, Baltimore · USJohns Hopkins University · USUniversity of Maryland, College Park · USRadiology Associates · USUniversity of Colorado Denver · USUniversity of Michigan · USYale University · US

Funding

Human-centered CT-based CADx Tools for Traumatic Torso HemorrhageR01GM148987 · NIGMS · UNIVERSITY OF MARYLAND BALTIMORE · PI David Dreizin · 2023 to 2026
$1.5M
Machine learning-based segmentation and risk modeling for real-time prediction of major arterial bleeding after pelvic fracturesK08EB027141 · NIBIB · UNIVERSITY OF MARYLAND BALTIMORE · PI DREIZIN, DAVID · 2019 to 2022
$745k
Foundation for the National Institutes of Health NIH-K08-EB027141-01A1Foundation for the National Institutes of Health NIH-R01-GM148987-01NIBIB NIH HHS K08 EB027141NIGMS NIH HHS R01 GM148987
6 · The paper itself

Abstract

purposeThe AAST Organ Injury Scale is widely adopted for splenic injury severity but suffers from only moderate inter-rater agreement. This work assesses SpleenPro, a prototype interactive explainable artificial intelligence/machine learning (AI/ML) diagnostic aid to support AAST grading, for effects on radiologist dwell time, agreement, clinical utility, and user acceptance.

methodsTwo trauma radiology ad hoc expert panelists independently performed timed AAST grading on 76 admission CT studies with blunt splenic injury, first without AI/ML assistance, and after a 2-month washout period and randomization, with AI/ML assistance. To evaluate user acceptance, three versions of the SpleenPro user interface with increasing explainability were presented to four independent expert panelists with four example cases each. A structured interview consisting of Likert scales and free responses was conducted, with specific questions regarding dimensions of diagnostic utility (DU); mental support (MS); effort, workload, and frustration (EWF); trust and reliability (TR); and likelihood of future use (LFU).

resultsSpleenPro significantly decreased interpretation times for both raters. Weighted Cohen's kappa increased from 0.53 to 0.70 with AI/ML assistance. During user acceptance interviews, increasing explainability was associated with improvement in Likert scores for MS, EWF, TR, and LFU. Expert panelists indicated the need for a combined early notification and grading functionality, PACS integration, and report autopopulation to improve DU.

conclusionsSpleenPro was useful for improving objectivity of AAST grading and increasing mental support. Formative user research identified generalizable concepts including the need for a combined detection and grading pipeline and integration with the clinical workflow.

Indexed as

Tomography, X-Ray ComputedWounds, NonpenetratingArtificial IntelligenceHumansMachine LearningReproducibility of ResultsAAST gradeArtificial intelligenceHemorrhageSpleenTraumaUser acceptance

Identifiers

PMID38302827
PMCPMC11257379
OpenAlexW4392785424

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.