Evidence map›Paper›PMID 34378444›Full record

ArticleMedical decision making : an international journal of the Society for Medical Decision Making2022

Comparison of Diagnostic Recommendations from Individual Physicians versus the Collective Intelligence of Multiple Physicians in Ambulatory Cases Referred for Specialist Consultation.

Elaine C Khoong, Sarah S Nouri, Delphine S Tuot, Shantanu Nundy, Valy Fontil, Urmimala Sarkar

Abstract read
In one paragraph

Article in Medical decision making : an international journal of the Society for Medical Decision Making, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 6 papers.

0numbers the graph read from it
0cells of the map it votes in
6citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

6 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Collective Intelligence Increases Diagnostic Accuracy in a General Practice Setting.Medical decision making : an international journal of the Society for Medical Decision Making · 2024
    Article
  5. Automating hybrid collective intelligence in open-ended medical diagnostics.Proceedings of the National Academy of Sciences of the United States of America · 2023
    Article
  6. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Elaine C KhoongDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, UCSF, San Francisco, CA, USA.ORCID 0000-0002-2514-3572
Sarah S NouriDivision of General Internal Medicine, Department of Medicine, UCSF, San Francisco, CA, USA.
Delphine S TuotCenter for Vulnerable Populations at Zuckerberg San Francisco General Hospital, UCSF, San Francisco, CA,USA.
Shantanu NundyGeorge Washington University Milken Institute School of Public Health, Washington, DC, USA.
Valy FontilDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, UCSF, San Francisco, CA, USA.
Urmimala SarkarDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, UCSF, San Francisco, CA, USA.

Funding

Clinical and Translational Science InstituteUL1TR001872 · NCATS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI COLLARD, HAROLD R, JACOBY, VANESSA · 2016 to 2025
$112.1M
Institutional Career Development Core SupplementKL2TR001870 · NCATS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI KANAYA, ALKA M., SARKAR, URMIMALA · 2016 to 2025
$19.7M
UCSF Career Development Program in Implementation Research in Heart and Lung DiseasesK12HL138046 · NHLBI · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI BIBBINS-DOMINGO, KIRSTEN, CATTAMANCHI, ADITHYA · 2017 to 2021
$2.9M
Improving Survivorship Care for Diverse Cancer Patients Cared for in Safety-net SettingsK24CA212294 · NCI · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI SARKAR, URMIMALA · 2017 to 2021
$831k
NCATS NIH HHS KL2 TR001870NCATS NIH HHS UL1 TR001872NCI NIH HHS K24 CA212294NHLBI NIH HHS K12 HL138046
6 · The paper itself

Abstract

backgroundStudies report higher diagnostic accuracy using the collective intelligence (CI) of multiple clinicians compared with individual clinicians. However, the diagnostic process is iterative, and unexplored is the value of CI in improving clinical recommendations leading to a final diagnosis.

methodsTo compare the appropriateness of diagnostic recommendations advised by individual physicians versus the CI of physicians, we entered actual consultation requests sent by primary care physicians to specialists onto a web-based CI platform capable of collecting diagnostic recommendations (next steps for care) from multiple physicians. We solicited responses to 35 cases (12 endocrinology, 13 gynecology, 10 neurology) from ≥3 physicians of any specialty through the CI platform, which aggregated responses into a CI output. The primary outcome was the appropriateness of individual physician recommendations versus the CI output recommendations, using recommendations agreed upon by 2 specialists in the same specialty as a gold standard. The secondary outcome was the recommendations' potential for harm.

resultsA total of 177 physicians responded. Cases had a median of 7 respondents (interquartile range: 5-10). Diagnostic recommendations in the CI output achieved higher levels of appropriateness (69%) than recommendations from individual physicians (45%; χ LIMITATIONS: Cases were from a single institution. CI was solicited using a single algorithm/platform.

conclusionsWhen seeking specialist guidance, diagnostic recommendations from the CI of multiple physicians are more appropriate than recommendations from most individual physicians, measured against specialist recommendations. Although CI provides useful recommendations, some have potential for harm. Future research should explore how to use CI to improve diagnosis while limiting harm from inappropriate tests/therapies.

Indexed as

PhysiciansAmbulatory Care FacilitiesHumansIntelligenceReferral and Consultationcollective intelligencediagnosisdiagnostic errorshealth information technology

Identifiers

PMID34378444
PMCPMC8831645

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.