Evidence map›Paper›PMID 41851268›Full record

ArticleNPJ digital medicine2026

From tool to teammate in a randomized controlled trial of clinician-AI collaborative workflows for diagnosis.

Selin S Everett, Bryan J Bunning, Priyank Jain, Ivan Lopez, Anup Agarwal, Manisha Desai, Robert Gallo, Ethan Goh, Vinay B Kadiyala, Zahir Kanjee and 7 more

Registry-linked trialAbstract read
In one paragraph

Article in NPJ digital medicine, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT06911645 (Evaluating the Performance of LLMs and Clinicians in Complex Diagnostic Cases), which is not on this map. Cited by 18 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
18citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT06911645 nacompletednot on this map

Evaluating the Performance of LLMs and Clinicians in Complex Diagnostic Cases: A Randomized Controlled Trial

TypeinterventionalSponsorStanford UniversityRan2024 to 2025Enrolled70ConditionsPathologic Processes, DiseaseArmsImmediate access to customized version of GPT-4, Access to customized version of GPT-4 following use of conventional resources
3 · Its place in the literature

Who cites it

18 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Trial
  3. Article
  4. The next frontier in healthcare simulation is human-AI teaming.Advances in simulation (London, England) · 2026
    Article
  5. Article
  6. Review
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. State of clinical AI in 2026.BMJ digital health & AI · 2026
    Review
  17. Prompt injection of OpenAI custom GPTs leaks informatics secrets.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2026
    Article
  18. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

17 authors.

Selin S EverettStanford University School of Medicine, Stanford, CA, USA.
Bryan J BunningStanford Department of Biomedical Data Science, Stanford, CA, USA.
Priyank JainHarvard Medical School, Boston, MA, USA.
Ivan LopezStanford University School of Medicine, Stanford, CA, USA.
Anup AgarwalHarvard Medical School, Boston, MA, USA.
Manisha DesaiStanford University School of Medicine, Stanford, CA, USA.
Robert GalloCenter for Innovation to Implementation, VA Palo Alto Health Care System, Palo Alto, CA, USA.
Ethan GohStanford Center for Biomedical Informatics Research, Stanford University, Stanford, CA, USA.
Vinay B KadiyalaHarvard Medical School, Boston, MA, USA.
Zahir KanjeeHarvard Medical School, Boston, MA, USA.
Jacob M KoshyHarvard Medical School, Boston, MA, USA.
Andrew OlsonUniversity of Minnesota Medical School, Minneapolis, MN, USA.
Adam RodmanHarvard Medical School, Boston, MA, USA.
Kevin SchulmanDivision of Hospital Medicine, Stanford University, Stanford, CA, USA.
Eric StrongDivision of Hospital Medicine, Stanford University, Stanford, CA, USA.
Jonathan H ChenStanford Center for Biomedical Informatics Research, Stanford University, Stanford, CA, USA.
Eric HorvitzOffice of the Chief Scientific Officer, Microsoft, Redmond, WA, USA. horvitz@microsoft.com.

Funding

Gordon and Betty Moore Foundation 12409U.S. National Library of Medicine 2T15LM007033
6 · The paper itself

Abstract

Early studies of large language models (LLMs) in clinical settings have largely treated artificial intelligence (AI) as a tool rather than an active collaborator. As LLMs demonstrate expert-level diagnostic performance, the focus shifts from whether AI can offer valuable suggestions to how it integrates into physicians' diagnostic workflows. We conducted a randomized controlled trial (n = 70 clinicians) to assess a custom system designed for collaborative diagnostic reasoning. The design involved independent diagnostic assessments by the clinician and AI, followed by an AI-generated synthesis integrating both perspectives, highlighting agreements, disagreements, and offering commentary. We evaluated two collaborative workflows: AI as first opinion (preceding clinician) and AI as second opinion (following clinician). Both improved clinician diagnostic accuracy over conventional resources, (85% and 82% vs. 75%). Performance was comparable across workflows and not statistically different from AI-alone accuracy (90%), highlighting the potential of collaborative AI to complement clinician expertise. Qualitative analyses illustrate how workflow design shapes human-AI interaction. C : NCT06911645.

Identifiers

PMID41851268
PMCPMC13222345

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.