ArticleBMC medical research methodology2025
Utilizing Large language models to select literature for meta-analysis shows workload reduction while maintaining a similar recall level as manual curation.
Article in BMC medical research methodology, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Artificial Intelligence for Evidence Synthesis of Emerging Biologics to Improve Skeletal Health in Osteogenesis Imperfecta: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2026Pooled it
- The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review.Journal of the American Medical Informatics Association : JAMIA · 2025Pooled it
- Artificial Intelligence Resources for the Screening of Titles and Abstracts in Systematic Reviews: A Scoping Review.Cochrane evidence synthesis and methods · 2026Review
- Applications of DeepSeek in Medicine: Bibliometric Analysis and Scoping Review.Journal of medical Internet research · 2026Article
- Evaluating Elicit's systematic reviews workflow in an umbrella review on air pollution and acute lower respiratory infections: a methodological study for quality appraisal.BMC medical research methodology · 2026Article
- CCMRI: a classification and curated database of climate change-related microbiome studies.Scientific reports · 2026Article
- Applications of Large Language Models in Medical Research: From Systematic Reviews to Clinical Studies.Bioengineering (Basel, Switzerland) · 2026Review
- Evaluating the Clinical Competence of Large Language Models in Prostate Cancer Management: A Comparative Study of DeepSeek-R1 and ChatGPT.Annals of surgical oncology · 2026Article
- Cyberbullying victimization identification and large language model-assisted assessment: a study of cyberbullying victimization lexicon construction and validation.Frontiers in psychiatry · 2026Article
- Harnessing ChatGPT for abstract screening in health-related scoping reviews: the role of structured eligibility criteria.BMC health services research · 2025Article
- Promoting Responsible DeepSeek Deployment in Health Care: Scoping Review Comparing Grey and White Literature.Journal of medical Internet research · 2025Article
- From promise to practice: challenges and pitfalls in the evaluation of large language models for data extraction in evidence synthesis.BMJ evidence-based medicine · 2025Article
- Automating inductive thematic analyses of health content using large language models: a proof-of-concept study using social media data.JAMIA open · 2025Article
- Positive sentiments in early academic literature on DeepSeek: a cross-disciplinary mini review.Frontiers in artificial intelligence · 2025Review
- Large language model enhanced framework for systematic reviews and meta-analyses.BMJ digital health & AI · 2025Article
- Development of a questionnaire for assessing the use of ChatGPT in primary and secondary disease prevention.Frontiers in public health · 2025Article
- Can machine learning help accelerate article screening for systematic reviews? Yes, when article separability in embedding space is high.Research synthesis methods · 2025Article
- Confronting the data deluge: How artificial intelligence can be used in the study of plant stress.Computational and structural biotechnology journal · 2024Review
- Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses.Journal of medical Internet research · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
backgroundLarge language models (LLMs) like ChatGPT showed great potential in aiding medical research. A heavy workload in filtering records is needed during the research process of evidence-based medicine, especially meta-analysis. However, few studies tried to use LLMs to help screen records in meta-analysis.
objectiveIn this research, we aimed to explore the possibility of incorporating multiple LLMs to facilitate the screening step based on the title and abstract of records during meta-analysis.
methodsVarious LLMs were evaluated, which includes GPT-3.5, GPT-4, Deepseek-R1-Distill, Qwen-2.5, Phi-4, Llama-3.1, Gemma-2 and Claude-2. To assess our strategy, we selected three meta-analyses from the literature, together with a glioma meta-analysis embedded in the study, as additional validation. For the automatic selection of records from curated meta-analyses, a four-step strategy called LARS-GPT was developed, consisting of (1) criteria selection and single-prompt (prompt with one criterion) creation, (2) best combination identification, (3) combined-prompt (prompt with one or more criteria) creation, and (4) request sending and answer summary. Recall, workload reduction, precision, and F1 score were calculated to assess the performance of LARS-GPT.
resultsA variable performance was found between different single-prompts, with a mean recall of 0.800. Based on these single-prompts, we were able to find combinations with better performance than the pre-set threshold. Finally, with a best combination of criteria identified, LARS-GPT showed a 40.1% workload reduction on average with a recall greater than 0.9.
conclusionsWe show here the groundbreaking finding that automatic selection of literature for meta-analysis is possible with LLMs. We provide it here as a pipeline, LARS-GPT, which showed a great workload reduction while maintaining a pre-set recall.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.