Evidence map›Paper›PMID 39918783›Full record

ReviewCJEM2025

Establishing methodological standards for the development of artificial intelligence-based Clinical Decision Support in emergency medicine.

Hashim Kareemi, Henry Li, Akshay Rajaram, Jessalyn K Holodinsky, Justin N Hall, Lars Grant, Gautam Goel, Jake Hayward, Shaun Mehta, Maxim Ben-Yakov and 3 more

Abstract readReview
PubMed Publisher
In one paragraph

Review in CJEM, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers.

0numbers the graph read from it
0cells of the map it votes in
5citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

5 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

13 authors.

Hashim KareemiDepartment of Emergency Medicine, University of British Columbia, Vancouver, BC, Canada. hashim.kareemi@vch.ca.ORCID 0000-0001-8599-1576
Henry LiDepartment of Emergency Medicine, University of Alberta, Edmonton, AB, Canada.
Akshay RajaramDepartment of Emergency Medicine, Queen's University, Kingston, ON, Canada.
Jessalyn K HolodinskyDepartment of Emergency Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
Justin N HallDepartment of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada.
Lars GrantDepartment of Emergency Medicine, McGill University, Montreal, QC, Canada.
Gautam GoelDepartment of Emergency Medicine, Queensway Carleton Hospital, Ottawa, ON, Canada.
Jake HaywardDepartment of Emergency Medicine, University of Alberta, Edmonton, AB, Canada.
Shaun MehtaDepartment of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada.
Maxim Ben-YakovDepartment of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada.
Elyse Berger PelletierDepartment of Family Medicine and Emergency Medicine, Université Laval, Quebec, QC, Canada.
Frank ScheuermeyerDepartment of Emergency Medicine, University of British Columbia, Vancouver, BC, Canada.
Kendall HoDepartment of Emergency Medicine, University of British Columbia, Vancouver, BC, Canada.

Funding

CIHR Canada Graduate Scholarship - Master's
6 · The paper itself

Abstract

objectiveArtificial intelligence (AI) offers opportunities for managing the complexities of clinical care in the emergency department (ED), and Clinical Decision Support has been identified as a priority application. However, there is a lack of published guidance on how to rigorously develop and evaluate these tools. We sought to answer the question, "What methodological standards should be applied to the development of AI-based Clinical Decision Support tools in the ED?".

methodsWe conducted an iterative consensus-establishing activity involving a subcommittee with AI expertise followed by surveys and a live facilitated discussion with participants of the 2024 Canadian Association of Emergency Physicians Research Symposium in Saskatoon. We augmented analysis of participant feedback with large language models.

resultsWe established 11 recommendations AI-based Clinical Decision Support development including the selection of a relevant problem and team of experts, standards of data quality and quantity, novel AI-specific reporting guidelines, and adherence to principles of ethics and privacy. We removed the recommendation regarding model interpretability from the final list due to a lack of consensus.

conclusionThese 11 recommendations provide guiding principles and methodological standards for emergency medicine researchers to rigorously develop AI-based Clinical Decision Support tools and for clinicians to gain knowledge and trust in using them.

Indexed as

Artificial IntelligenceDecision Support Systems, ClinicalEmergency MedicineEmergency Service, HospitalCanadaConsensusHumansArtificial intelligenceEmergency medicineMachine learningMethodologyResearch

Identifiers

PMID39918783

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.