Evidence map›Paper›PMID 39999154›Full record

ArticleJMIR human factors2025

Comparison of a Novel Machine Learning-Based Clinical Query Platform With Traditional Guideline Searches for Hospital Emergencies: Prospective Pilot Study of User Experience and Time Efficiency.

Hamza Ejaz, Hon Lung Keith Tsui, Mehul Patel, Luis Rafael Ulloa Paredes, Ellen Knights, Shah Bakht Aftab, Christian Peter Subbe

Abstract readComparative Study
In one paragraph

Article in JMIR human factors, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.

0numbers the graph read from it
0cells of the map it votes in
1citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

1 citing paper in PubMed.

  1. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Hamza EjazYsbyty Gwynedd, Clinical School, Clinical Research Office, Bangor, United Kingdom.ORCID 0000-0001-7722-1774
Hon Lung Keith Tsui *Medwise.ai Ltd, Leeds, United Kingdom.ORCID 0000-0003-3945-1450
Mehul PatelMedwise.ai Ltd, Leeds, United Kingdom.ORCID 0009-0009-6476-3924
Luis Rafael Ulloa ParedesMedwise.ai Ltd, Leeds, United Kingdom.ORCID 0009-0009-8252-154X
Ellen KnightsYsbyty Gwynedd, Clinical School, Clinical Research Office, Bangor, United Kingdom.ORCID 0000-0001-7759-1732
Shah Bakht Aftab *Russells Hall Hospital, Dudley, United Kingdom.ORCID 0009-0005-4539-8758
Christian Peter Subbe *4, North Wales Medical School, Bangor University, Brigantia Building, Bangor, LL57 2AS, United Kingdom, 44 7771922890.ORCID 0000-0002-3110-8888

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Unlabelled: Background: Emergency and acute medicine doctors require easily accessible evidence-based information to safely manage a wide range of clinical presentations. The inability to find evidence-based local guidelines on the trust's intranet leads to information retrieval from the World Wide Web. Artificial intelligence (AI) has the potential to make evidence-based information retrieval faster and easier. Objective: The aim of the study is to conduct a time-motion analysis, comparing cohorts of junior doctors using (1) an AI-supported search engine versus (2) the traditional hospital intranet. The study also aims to examine the impact of the AI-supported search engine on the duration of searches and workflow when seeking answers to clinical queries at the point of care. Methods: This pre- and postobservational study was conducted in 2 phases. In the first phase, clinical information searches by 10 doctors caring for acutely unwell patients in acute medicine were observed during 10 working days. Based on these findings and input from a focus group of 14 clinicians, an AI-supported, context-sensitive search engine was implemented. In the second phase, clinical practice was observed for 10 doctors for an additional 10 working days using the new search engine. Results: The hospital intranet group (n=10) had a median of 23 months of clinical experience, while the AI-supported search engine group (n=10) had a median of 54 months. Participants using the AI-supported engine conducted fewer searches. User satisfaction and query resolution rates were similar between the 2 phases. Searches with the AI-supported engine took 43 seconds longer on average. Clinicians rated the new app with a favorable Net Promoter Score of 20. Conclusions: We report a successful feasibility pilot of an AI-driven search engine for clinical guidelines. Further development of the engine including the incorporation of large language models might improve accuracy and speed. More research is required to establish clinical impact in different user groups. Focusing on new staff at beginning of their post might be the most suitable study design.

Indexed as

Emergency Service, HospitalMachine LearningSearch EngineHumansPilot ProjectsProspective StudiesTime and Motion Studiesartificial intelligenceclinical experienceclinical impactclinical practicedevelopingemergency carehospital careinformation retrievalinformation searchmachine learningmobile phonestudy designtestingtraininguser groupusersuser satisfaction

Identifiers

PMID39999154
PMCPMC11878475

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.