Evidence map›Paper›PMID 33568741›Full record

ArticleCommunications biology2021

Systematic auditing is essential to debiasing machine learning in biology.

Fatma-Elzahraa Eid, Haitham A Elmarakeby, Yujia Alina Chan, Nadine Fornelos, Mahmoud ElHefnawi, Eliezer M Van Allen, Lenwood S Heath, Kasper Lage

Abstract read
In one paragraph

Article in Communications biology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 20 papers.

0numbers the graph read from it
0cells of the map it votes in
20citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

20 citing papers in PubMed.

  1. From patient notes to prognostication: The revolutionary potential of event-based foundation models in orthopaedics.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026
    Article
  2. Article
  3. Review
  4. Review
  5. Article
  6. Review
  7. Bias-aware training and evaluation of link prediction algorithms in network biology.Proceedings of the National Academy of Sciences of the United States of America · 2025
    Article
  8. Review
  9. Review
  10. Review
  11. Article
  12. Advancing age grading techniques forBiology methods & protocols · 2024
    Article
  13. Article
  14. Article
  15. Review
  16. Article
  17. Review
  18. Review
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

8 authors.

Fatma-Elzahraa Eid *Broad Institute of MIT and Harvard, Cambridge, MA, USA. fatma@broadinstitute.org.ORCID 0000-0002-6822-9536
Haitham A Elmarakeby *Broad Institute of MIT and Harvard, Cambridge, MA, USA.
Yujia Alina Chan *Broad Institute of MIT and Harvard, Cambridge, MA, USA.ORCID 0000-0002-0731-637X
Nadine FornelosBroad Institute of MIT and Harvard, Cambridge, MA, USA.
Mahmoud ElHefnawiInformatics and Systems Department, Division of Engineering Research, National Research Centre, Giza, Egypt.
Eliezer M Van AllenBroad Institute of MIT and Harvard, Cambridge, MA, USA.
Lenwood S HeathVirginia Polytechnic Institute and State University, Blacksburg, VA, USA.ORCID 0000-0003-1608-431X
Kasper LageBroad Institute of MIT and Harvard, Cambridge, MA, USA. lage.kasper@mgh.harvard.edu.ORCID 0000-0001-6827-6239

Funding

2/3 Building Integrative CNS Networks for Genomic Analysis of AutismR01MH109903 · NIMH · MASSACHUSETTS GENERAL HOSPITAL · PI HANSEN, KASPER LAGE · 2016 to 2020
$1.4M
NIMH NIH HHS R01 MH109903
6 · The paper itself

Abstract

Biases in data used to train machine learning (ML) models can inflate their prediction performance and confound our understanding of how and what they learn. Although biases are common in biological data, systematic auditing of ML models to identify and eliminate these biases is not a common practice when applying ML in the life sciences. Here we devise a systematic, principled, and general approach to audit ML models in the life sciences. We use this auditing framework to examine biases in three ML applications of therapeutic interest and identify unrecognized biases that hinder the ML process and result in substantially reduced model performance on new datasets. Ultimately, we show that ML models tend to learn primarily from data biases when there is insufficient signal in the data to learn from. We provide detailed protocols, guidelines, and examples of code to enable tailoring of the auditing framework to other biomedical applications.

Indexed as

Data MiningMachine LearningProteomeProteomicsAnimalsBiasDatabases, ProteinHistocompatibility AntigensHumansPharmaceutical PreparationsProtein BindingProtein Interaction MapsProteinsReproducibility of ResultsHistocompatibility AntigensPharmaceutical PreparationsProteinsProteome

Identifiers

PMID33568741
PMCPMC7876113

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.