Evidence map›Paper›PMID 40964933›Full record

ArticleBioethics2026

When Is It Safe to Introduce an AI System Into Healthcare? A Practical Decision Algorithm for the Ethical Implementation of Black-Box AI in Medicine.

Jemima Winifred Allen, Dominic Wilkinson, Julian Savulescu

Abstract read
In one paragraph

Article in Bioethics, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers.

0numbers the graph read from it
0cells of the map it votes in
5citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

5 citing papers in PubMed.

  1. Accountability for large language models in health care.Bulletin of the World Health Organization · 2026
    Article
  2. Review
  3. Article
  4. Article
  5. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Jemima Winifred AllenDepartment of Paediatrics, Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, Victoria, Australia.ORCID 0000-0002-3950-9950
Dominic WilkinsonDepartment of Paediatrics, Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, Victoria, Australia.ORCID 0000-0003-3958-8633
Julian SavulescuUehiro Oxford Insitute, University of Oxford, Oxford, UK.

Funding

This study/project is supported by the National Research Foundation, Singapore, under its AI Singapore Programme (AISG Award No: AISG3-GV-2023-012). Additionally, this study was funded in part by the Wellcome Trust [203132/Z/16/Z]. This study was supported by the Wellcome Trust [Grant number: 226801] for the Discovery Research Platform for Transformative Inclusivity in Ethics and Humanities Research (ANTITHESES). The funders had no role in the preparation of this manuscript or the decision to submit for publication. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.Wellcome Trust
6 · The paper itself

Abstract

There is mounting global interest in the revolutionary potential of AI tools. However, its use in healthcare carries certain risks. Some argue that opaque ('black box') AI systems in particular undermine patients' informed consent. While interpretable models offer an alternative, this approach may be impossible with generative AI and large language models (LLMs). Thus, we propose that AI tools should be evaluated for clinical use based on their implementation risk, rather than interpretability. We introduce a practical decision algorithm for the clinical implementation of black-box AI by evaluating its risk of implementation. Applied to the case of an LLM for surgical informed consent, we assess a system's implementation risk by evaluating: (1) technical robustness, (2) implementation feasibility and (3) analysis of harms and benefits. Accordingly, the system is categorised as minimal-risk (standard use), moderate-risk (innovative use) or high-risk (experimental use). Recommendations for implementation are proportional to risk, requiring more oversight for higher-risk categories. The algorithm also considers the system's cost-effectiveness and patients' informed consent.

Indexed as

AlgorithmsArtificial IntelligenceDelivery of Health CareInformed ConsentCost-Benefit AnalysisHumansRisk Assessmentartificial intelligenceblack‐box AIclinical practiceinformed consentlarge language modelsrisk assessment

Identifiers

PMID40964933
PMCPMC12710678

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.