Evidence map›Paper›PMID 35477730›Full record

ArticleScientific reports2022

Differences between human and machine perception in medical diagnosis.

Taro Makino, Stanisław Jastrzębski, Witold Oleszkiewicz, Celin Chacko, Robin Ehrenpreis, Naziya Samreen, Chloe Chhor, Eric Kim, Jiyon Lee, Kristine Pysarenko and 11 more

Abstract read
In one paragraph

Article in Scientific reports, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 7 papers.

0numbers the graph read from it
0cells of the map it votes in
7citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

7 citing papers in PubMed.

  1. Article
  2. Review
  3. Article
  4. Article
  5. Article
  6. Review
  7. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

21 authors.

Taro MakinoCenter for Data Science, New York University, New York, NY, USA. taro@nyu.edu.
Stanisław JastrzębskiCenter for Data Science, New York University, New York, NY, USA.
Witold OleszkiewiczFaculty of Electronics and Information Technology, Warsaw University of Technology, Warszawa, Poland.
Celin ChackoDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Robin EhrenpreisDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Naziya SamreenDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Chloe ChhorDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Eric KimDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Jiyon LeeDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Kristine PysarenkoDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Beatriu ReigDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Hildegard TothDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Divya AwalDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Linda DuDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Alice KimDepartment of Radiology, NYU Langone Health, New York, NY, USA.
James ParkDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Daniel K SodicksonDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Laura HeacockDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Linda MoyDepartment of Radiology, NYU Langone Health, New York, NY, USA.
Kyunghyun ChoCenter for Data Science, New York University, New York, NY, USA.
Krzysztof J GerasCenter for Data Science, New York University, New York, NY, USA. k.j.geras@nyu.edu.

Funding

TR&D 4: Revealing Microstructure: Biophysical modeling and validation for discovery and clinical careP41EB017183 · NIBIB · NEW YORK UNIVERSITY SCHOOL OF MEDICINE · PI Christopher M Collins · 2014 to 2026
$19.3M
Reducing recall rates of screening mammography with deep convolutional neural networkR21CA225175 · NCI · NEW YORK UNIVERSITY SCHOOL OF MEDICINE · PI MOY, LINDA · 2018 to 2019
$395k
NCI NIH HHS R21 CA225175NIBIB NIH HHS P41 EB017183NIH HHS P41EB017183NIH HHS R21CA225175
6 · The paper itself

Abstract

Deep neural networks (DNNs) show promise in image-based medical diagnosis, but cannot be fully trusted since they can fail for reasons unrelated to underlying pathology. Humans are less likely to make such superficial mistakes, since they use features that are grounded on medical science. It is therefore important to know whether DNNs use different features than humans. Towards this end, we propose a framework for comparing human and machine perception in medical diagnosis. We frame the comparison in terms of perturbation robustness, and mitigate Simpson's paradox by performing a subgroup analysis. The framework is demonstrated with a case study in breast cancer screening, where we separately analyze microcalcifications and soft tissue lesions. While it is inconclusive whether humans and DNNs use different features to detect microcalcifications, we find that for soft tissue lesions, DNNs rely on high frequency components ignored by radiologists. Moreover, these features are located outside of the region of the images found most suspicious by radiologists. This difference between humans and machines was only visible through subgroup analysis, which highlights the importance of incorporating medical domain knowledge into the comparison.

Indexed as

Breast NeoplasmsCalcinosisFemaleHumansNeural Networks, ComputerPerceptionRadiologists

Identifiers

PMID35477730
PMCPMC9046399

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.