Evidence map›Paper›PMID 39701919›Full record

SynthesisThe Lancet. Digital health2025

Tackling algorithmic bias and promoting transparency in health datasets: the STANDING Together consensus recommendations.

Joseph E Alderman, Joanne Palmer, Elinor Laws, Melissa D McCradden, Johan Ordish, Marzyeh Ghassemi, Stephen R Pfohl, Negar Rostamzadeh, Heather Cole-Lewis, Ben Glocker and 40 more

Abstract readSystematic ReviewConsensus Statement
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in The Lancet. Digital health, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 97 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
97citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

97 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. Turing problems in otolaryngology: a scoping review of the principal challenges of artificial ıntelligence and large language models.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026
    Review
  5. Review
  6. Article
  7. Review
  8. Article
  9. Towards Human-Centered Digital Health Interventions.The Psychiatric clinics of North America · 2026
    Review
  10. Review
  11. Review
  12. Review
  13. Review
  14. Review
  15. Article
  16. Article
  17. Review
  18. Review
  19. Article
  20. Article

37 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

50 authors.

Joseph E AldermanUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK; University of Birmingham, Birmingham, UK.
Joanne PalmerUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK; University of Birmingham, Birmingham, UK.
Elinor LawsUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK; University of Birmingham, Birmingham, UK.
Melissa D McCraddenDepartment of Bioethics, The Hospital for Sick Children, Toronto, ON, Canada; Genetics and Genome Biology, SickKids Research Institute, Toronto, ON, Canada.
Johan OrdishUniversity of Birmingham, Birmingham, UK; Roche Diagnostics, Rotkreuz, Switzerland; Hughes Hall, Cambridge, UK.
Marzyeh GhassemiDepartment of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA; Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
Stephen R PfohlGoogle Research, Mountain View, CA, USA.
Negar RostamzadehGoogle Research, Montreal, Quebec, USA.
Heather Cole-LewisGoogle, Mountain View, CA, USA.
Ben GlockerDepartment of Computing, Imperial College London, London, UK.
Melanie CalvertUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, Birmingham, UK; NIHR Applied Research Collaboration West Midlands, Birmingham, UK; NIHR Blood and Transplant Research Unit in Precision Transplant and Cellular Therapeutics, Birmingham, UK.
Tom J PollardInstitute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
Jaspret GillUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
Jacqui GathIndependent Cancer Patients' Voice, London, UK; Patient and Public Contributor, Sheffield, UK.
Adewale AdebajoNIHR Sheffield Biomedical Research Centre, Sheffield, UK.
Jude BengSchool of Medicine and Population Health, University of Sheffield, Sheffield, UK.
Cassandra H Leungpatient and public contributor, Munich, Germany.
Stephanie KukuInstitute of Women's Health, University College London, London, UK.
Lesley-Anne FarmerTethyan Consulting, Canberra, Australia.
Rubeta N MatinOxford University Hospitals NHS Foundation Trust, Oxford, UK; University of Oxford, Oxford, UK.
Bilal A MateenInstitute of Health Informatics, University College London, London, UK; PATH, Seattle, WA, USA; Wellcome Trust, London, UK.
Francis McKayPopulation Health Sciences Institute, Newcastle University, Newcastle, UK; Health Determinants Research Collaboration, Gateshead Council, Gateshead, UK.
Katherine HellerGoogle Research, Mountain View, CA, USA.
Alan KarthikesalingamGoogle, London, UK.
Darren TreanorLeeds Teaching Hospitals NHS Trust, Leeds, UK; University of Leeds, Leeds, UK; Department of Clinical Pathology and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden.
Maxine MackintoshGenomics England, London, UK; The Alan Turing Institute, London, UK.
Lauren Oakden-RaynerAustralian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia.
Russell PearsonMedicines and Healthcare products Regulatory Agency, London, UK.
Arjun K ManraiDepartment of Biomedical Informatics, Harvard Medical School, Harvard University, Cambridge, MA, USA.
Puja MylesMedicines and Healthcare products Regulatory Agency, London, UK.
Judit KumuthiniAfrican Biobanks and Longitudinal Epidemiologic Ecosystem, Ibadan, Nigeria.
Zoher KapaceeHealth Research Authority, London, UK.
Neil J SebireNIHR Great Ormond Street Hospital Biomedical Research Centre at UCL, University College London, London, UK.
Lama H NazerKing Hussein Cancer Centre, Amman, Jordan.
Jarrel SeahHarrison.ai, Sydney, NSW, Australia; Alfred Health, Melbourne, VIC, Australia; Monash University, Melbourne, VIC, Australia.
Ashley AkbariPopulation Data Science, Swansea University Medical School, Faculty of Medicine, Health and Life Science, Swansea University, Wales, UK.
Lew BermanAll of Us Research Program, National Institutes of Health, Office of the Director, Bethesda, MD, USA.
Judy W GichoyaDepartment of Radiology, Emory University School of Medicine, Atlanta, GA, USA.
Lorenzo RighettoNature Research, London, UK.
Diana SamuelThe Lancet Digital Health, The Lancet, London, UK.
William WasswaDepartment of Biomedical Sciences and Engineering, Mbarara University of Science and Technology, Mbarara, Uganda.
Maria CharalambidesDermatopharmacology, Faculty of Medicine, Southampton, UK; Department of Dermatology, University Hospitals Southampton NHS Foundation Trust, Southampton, UK.
Anmol AroraSchool of Clinical Medicine, Cambridge, UK; University of Cambridge, Cambridge, UK.
Sameer PujariWHO, Geneva, Switzerland.
Charlotte SummersVictor Phillip Dahdaleh Heart and Lung Research Institute, Cambridge, UK.
Elizabeth SapeyUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK; Centre for Patient Reported Outcomes Research, School of Health Sciences, College of Medical and Dental Sciences, Birmingham, UK; University of Birmingham, Birmingham, UK; PIONEER, HDR UK Health Data Hub in Acute Care, Birmingham, UK; NIHR Midlands Applied Research Collaboration, Acute Care Theme, West Midlands, UK; NIHR Midlands Patient Safety Collaboration, Birmingham, UK.
Sharon WilkinsonUniversity of Southampton, Southampton, UK; National Institute for Health and Care Research, Southampton, UK.
Vishal ThakkerBritish Standards Institute, London, UK.
Alastair DennistonUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, Birmingham, UK; Centre for Patient Reported Outcomes Research, School of Health Sciences, College of Medical and Dental Sciences, Birmingham, UK; University of Birmingham, Birmingham, UK; NIHR Biomedical Research Centre, Moorfields Eye Hospital and University College London, London, UK.
Xiaoxuan LiuUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK; Centre for Patient Reported Outcomes Research, School of Health Sciences, College of Medical and Dental Sciences, Birmingham, UK; University of Birmingham, Birmingham, UK. Electronic address: x.liu.8@bham.ac.uk.

Funding

World Health Organization 001
6 · The paper itself

Abstract

Without careful dissection of the ways in which biases can be encoded into artificial intelligence (AI) health technologies, there is a risk of perpetuating existing health inequalities at scale. One major source of bias is the data that underpins such technologies. The STANDING Together recommendations aim to encourage transparency regarding limitations of health datasets and proactive evaluation of their effect across population groups. Draft recommendation items were informed by a systematic review and stakeholder survey. The recommendations were developed using a Delphi approach, supplemented by a public consultation and international interview study. Overall, more than 350 representatives from 58 countries provided input into this initiative. 194 Delphi participants from 25 countries voted and provided comments on 32 candidate items across three electronic survey rounds and one in-person consensus meeting. The 29 STANDING Together consensus recommendations are presented here in two parts. Recommendations for Documentation of Health Datasets provide guidance for dataset curators to enable transparency around data composition and limitations. Recommendations for Use of Health Datasets aim to enable identification and mitigation of algorithmic biases that might exacerbate health inequalities. These recommendations are intended to prompt proactive inquiry rather than acting as a checklist. We hope to raise awareness that no dataset is free of limitations, so transparent communication of data limitations should be perceived as valuable, and absence of this information as a limitation. We hope that adoption of the STANDING Together recommendations by stakeholders across the AI health technology lifecycle will enable everyone in society to benefit from technologies which are safe and effective.

Indexed as

Delphi TechniqueAlgorithmsArtificial IntelligenceBiasDatasets as TopicHumans

Identifiers

PMID39701919
PMCPMC11668905

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.