Evidence map›Paper›PMID 36510176›Full record

ArticleBMC health services research2022

Artificial intelligence-enhanced care pathway planning and scheduling system: content validity assessment of required functionalities.

Miia Jansson, Pasi Ohtonen, Timo Alalääkkölä, Juuso Heikkinen, Minna Mäkiniemi, Sanna Lahtinen, Riikka Lahtela, Merja Ahonen, Sirpa Jämsä, Janne Liisantti

Abstract read
In one paragraph

Article in BMC health services research, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
11citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

11 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. Review
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Miia JanssonResearch Unit of Medical Imaging, Physics and Technology, University of Oulu, Oulu, Finland. miia.jansson@oulu.fi.ORCID http://orcid.org/0000-0001-5815-0325
Pasi OhtonenResearch Unit of Surgery, Anesthesia and Intensive Care, Oulu University Hospital, University of Oulu, Oulu, Finland.
Timo AlalääkköläTesting and Innovations, Oulu University Hospital, Oulu, Finland.ORCID http://orcid.org/0000-0003-1089-5357
Juuso HeikkinenDivision of Orthopedic and Trauma Surgery, Department of Surgery, Medical Research Center, Oulu University Hospital, Oulu, Finland.ORCID http://orcid.org/0000-0001-9137-788X
Minna MäkiniemiOulu University Hospital, Oulu, Finland.
Sanna LahtinenDepartment of Anesthesiology, Oulu University Hospital, Oulu, Finland.
Riikka LahtelaDepartment of Anesthesiology, Oulu University Hospital, Oulu, Finland.
Merja AhonenDepartment of Anesthesiology, Oulu University Hospital, Oulu, Finland.
Sirpa JämsäSense Organ Diseases Centre, Oulu University Hospital, Oulu, Finland.
Janne LiisanttiDepartment of Anesthesiology, Oulu University Hospital, Oulu, Finland.

Funding

This study was supported by the European Union's Horizon 2020 research and innovation program (nº 101016902) which is also gratefully acknowledged. nº 101016902
6 · The paper itself

Abstract

backgroundArtificial intelligence (AI) and machine learning are transforming the optimization of clinical and patient workflows in healthcare. There is a need for research to specify clinical requirements for AI-enhanced care pathway planning and scheduling systems to improve human-AI interaction in machine learning applications. The aim of this study was to assess content validity and prioritize the most relevant functionalities of an AI-enhanced care pathway planning and scheduling system.

methodsA prospective content validity assessment was conducted in five university hospitals in three different countries using an electronic survey. The content of the survey was formed from clinical requirements, which were formulated into generic statements of required AI functionalities. The relevancy of each statement was evaluated using a content validity index. In addition, weighted ranking points were calculated to prioritize the most relevant functionalities of an AI-enhanced care pathway planning and scheduling system.

resultsA total of 50 responses were received from clinical professionals from three European countries. An item-level content validity index ranged from 0.42 to 0.96. 45% of the generic statements were considered good. The highest ranked functionalities for an AI-enhanced care pathway planning and scheduling system were related to risk assessment, patient profiling, and resources. The highest ranked functionalities for the user interface were related to the explainability of machine learning models.

conclusionThis study provided a comprehensive list of functionalities that can be used to design future AI-enhanced solutions and evaluate the designed solutions against requirements. The relevance of statements concerning the AI functionalities were considered somewhat relevant, which might be due to the low level or organizational readiness for AI in healthcare.

Indexed as

Artificial IntelligenceCritical PathwaysDrugs, GenericHealth FacilitiesHumansMachine LearningProspective StudiesDrugs, GenericAICare pathwayhuman-AI interactionHuman-in-loopMachine learning

Identifiers

PMID36510176
PMCPMC9746075

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.