SynthesisInternational journal for equity in health2025
Evaluating and addressing demographic disparities in medical large language models: a systematic review.
Synthesis in International journal for equity in health, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 53 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
53 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Applications of natural language processing and large language models in sports injury assessment and rehabilitation decision-making: a scoping review.Frontiers in medicine · 2026Pooled it
- Large language models in healthcare: applications, evaluation frameworks, and governance pathways - a scoping review and multidimensional framework.Frontiers in digital health · 2026Pooled it
- A systematic review of ethical considerations of large language models in healthcare and medicine.Frontiers in digital health · 2025Pooled it
- Large Language Models and Empathy: Systematic Review.Journal of medical Internet research · 2024Pooled it
- Emerging applications of NLP and large language models in gastroenterology and hepatology: a systematic review.Frontiers in medicine · 2024Pooled it
- Cultural bias in large language models' ability to follow neuroradiology guidelines.European radiology · 2026Article
- Sex and gender bias in large language models: an old problem at a new scale.BMJ health & care informatics · 2026Article
- Large Language Models for Clinical Data Extraction in Workplace Injury Rehabilitation: Protocol for a Retrospective Pilot Study of Accuracy, Fairness, and Methodological Considerations in Workers' Compensation Medical Chart Review.JMIR research protocols · 2026Article
- Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review.JMIR research protocols · 2026Article
- Social Status and Clinical Resource Allocation by a Large Language Model: An Evaluation of 30,618 Decisions.Journal of personalized medicine · 2026Article
- Generative AI, foundation models and large language models in radiation therapy physics: Clinical applications, challenges, and future directions.Medical physics · 2026Review
- Fine-Tuning, Retrieval-Augmented Generation, and Hybrid Large Language Models for Postoperative Decision Support: Comparative Analysis.Journal of medical Internet research · 2026Article
- Evaluating Source-Based Large Language Models for Preclinical Dermatology Education: Comparative Study.JMIR formative research · 2026Article
- Ethical Considerations in Personal Health Large Language Models.Journal of medical Internet research · 2026Article
- How to meaningfully evaluate AI in clinical medicine.Nature medicine · 2026Article
- Impact of Query Language on the Structure and Guideline Alignment of AI-Generated Rehabilitation Programs in Chronic Kidney Disease.Journal of clinical medicine · 2026Article
- Demographic prompt cues shift clinical recommendations in multimodal large language models: a multi-model audit of 25,380 dental radiograph assessments.BMC oral health · 2026Article
- Algorithmic bias in clinical resource allocation by a large language model: a cross-sectional in-silico evaluation of 13,608 decisions.BMC medical ethics · 2026Article
- Bias Detection in Emergency Psychiatry: Linking Negative Language to Diagnostic Disparities.ArXiv · 2026Article
- New model, old risks: sociodemographic bias and adversarial hallucinations vulnerability in GPT-5.NPJ digital medicine · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundLarge language models are increasingly evaluated for use in healthcare. However, concerns about their impact on disparities persist. This study reviews current research on demographic biases in large language models to identify prevalent bias types, assess measurement methods, and evaluate mitigation strategies.
methodsWe conducted a systematic review, searching publications from January 2018 to July 2024 across five databases. We included peer-reviewed studies evaluating demographic biases in large language models, focusing on gender, race, ethnicity, age, and other factors. Study quality was assessed using the Joanna Briggs Institute Critical Appraisal Tools.
resultsOur review included 24 studies. Of these, 22 (91.7%) identified biases. Gender bias was the most prevalent, reported in 15 of 16 studies (93.7%). Racial or ethnic biases were observed in 10 of 11 studies (90.9%). Only two studies found minimal or no bias in certain contexts. Mitigation strategies mainly included prompt engineering, with varying effectiveness. However, these findings are tempered by a potential publication bias, as studies with negative results are less frequently published.
conclusionBiases are observed in large language models across various medical domains. While bias detection is improving, effective mitigation strategies are still developing. As LLMs increasingly influence critical decisions, addressing these biases and their resultant disparities is essential for ensuring fair artificial intelligence systems. Future research should focus on a wider range of demographic factors, intersectional analyses, and non-Western cultural contexts.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.