ReviewNPJ digital medicine2023
How AI can learn from the law: putting humans in the loop only on appeal.
Review in NPJ digital medicine, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed.
- Artificial Intelligence for Assessment and Feedback in Medical Education: Bibliometric Mapping Study and Thematic Evidence Map.JMIR medical education · 2026Article
- Prompt sensitivity of large language models in orthodontic patient counseling: a scenario-based experimental study.BMC oral health · 2026Article
- Real World Human-LLM Interactions - Prospective blinded versus unblinded expert physician assessments of LLM responses to complex medical dilemmas.PLOS digital health · 2026Article
- Explainable deep learning framework incorporating medical knowledge for insulin titration in diabetes.Communications medicine · 2026Article
- Construction and prototype effect evaluation of a multi-agent collaborative system for operating room nursing.Frontiers in digital health · 2026Article
- Justifiability and AI: putting explainability in its place.AI & society · 2026Article
- AI without representation is just inequity at scale: on the exportation of unrepresentative artificial intelligence models to the Global South.Frontiers in artificial intelligence · 2026Article
- Comparative evaluation of a hybrid machine learning-human adjudication paradigm for endoscopic scoring in ulcerative colitis.BMJ open gastroenterology · 2025Article
- Collaborative framework on responsible AI in LLM-driven CDSS for precision oncology leveraging real-world patient data.NPJ precision oncology · 2025Article
- How AI Is Transforming Medical Education: Bibliometric Analysis.JMIR medical education · 2025Article
- The Development and Evaluation of a Retrieval-Augmented Generation Large Language Model Virtual Assistant for Postoperative Instructions.Bioengineering (Basel, Switzerland) · 2025Article
- Applications, Challenges, and Prospects of Generative Artificial Intelligence Empowering Medical Education: Scoping Review.JMIR medical education · 2025Article
- Evaluation and Regulation of Artificial Intelligence Medical Devices for Clinical Decision Support.Annual review of biomedical data science · 2025Review
- Reasons underdetermination in meaningful human control.Ethics and information technology · 2025Article
- Personalized dose selection for the first Waldenström macroglobulinemia patient on the PRECISE CURATE.AI trial.NPJ digital medicine · 2024Article
- New regulatory thinking is needed for AI-based personalised drug and cell therapies in precision oncology.NPJ precision oncology · 2024Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
While the literature on putting a "human in the loop" in artificial intelligence (AI) and machine learning (ML) has grown significantly, limited attention has been paid to how human expertise ought to be combined with AI/ML judgments. This design question arises because of the ubiquity and quantity of algorithmic decisions being made today in the face of widespread public reluctance to forgo human expert judgment. To resolve this conflict, we propose that human expert judges be included via appeals processes for review of algorithmic decisions. Thus, the human intervenes only in a limited number of cases and only after an initial AI/ML judgment has been made. Based on an analogy with appellate processes in judiciary decision-making, we argue that this is, in many respects, a more efficient way to divide the labor between a human and a machine. Human reviewers can add more nuanced clinical, moral, or legal reasoning, and they can consider case-specific information that is not easily quantified and, as such, not available to the AI/ML at an initial stage. In doing so, the human can serve as a crucial error correction check on the AI/ML, while retaining much of the efficiency of AI/ML's use in the decision-making process. In this paper, we develop these widely applicable arguments while focusing primarily on examples from the use of AI/ML in medicine, including organ allocation, fertility care, and hospital readmission.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.