SynthesisResearch synthesis methods2025
Generative artificial intelligence use in evidence synthesis: A systematic review.
Synthesis in Research synthesis methods, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 22 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
22 citing papers in PubMed, 1 synthesis or guideline pooled it.
- A Critical Systematic Review of Modelling Approaches and Methodologies used in Hyperlipidaemia Economic Evaluations.PharmacoEconomics · 2026Pooled it
- Moving towards acceleration with accountability: a conceptual framework for AI-assisted systematic reviews.Global epidemiology · 2026Article
- Artificial Intelligence Resources for the Screening of Titles and Abstracts in Systematic Reviews: A Scoping Review.Cochrane evidence synthesis and methods · 2026Review
- Expanding the Scope of Prompt Engineering Training for Information Specialists in Evidence Synthesis.Campbell systematic reviews · 2026Article
- A Machine Learning and Large Language Model Tool for Systematic Literature Reviews of Health Economic Evidence: A Validation Study.PharmacoEconomics · 2026Article
- Evidence-to-Decision Frameworks: Enhancing the Quality and Rigour of Guidelines and Recommendations.Clinical and public health guidelines · 2026Article
- Using full agreement across multiple large language models for title-and-abstract screening in systematic reviews: a proof-of-concept.Systematic reviews · 2026Article
- Information Specialist Roles in the Era of Large Language Models: Prompting Continued Professional Development.Campbell systematic reviews · 2026Article
- Evaluating Elicit's systematic reviews workflow in an umbrella review on air pollution and acute lower respiratory infections: a methodological study for quality appraisal.BMC medical research methodology · 2026Article
- Artificial Intelligence Tools for Automating Evidence Synthesis: Scoping Review.Journal of medical Internet research · 2026Article
- Human-AI collaboration enhances the performance of large language models in risk of bias assessment.BMC medical research methodology · 2026Article
- Development and validation of search hedges for Transgender and Gender Diverse (TGD) populations in Ovid MEDLINE and Ovid APA PsycInfo.PloS one · 2026Article
- AI-assisted assessment of the IFSO consensus on obesity management medications in the context of metabolic bariatric surgery.PLOS digital health · 2025Article
- Promoting informed health choices: the long and winding road.Journal of the Royal Society of Medicine · 2025Article
- Assessing the Feasibility and Acceptability of a Bespoke Large Language Model Pipeline to Extract Data From Different Study Designs for Public Health Evidence Reviews.Cochrane evidence synthesis and methods · 2025Article
- Responsible Integration of Artificial Intelligence in Rapid Reviews: A Position Statement From the Cochrane Rapid Reviews Methods Group.Cochrane evidence synthesis and methods · 2025Article
- Enhancing Evidence Synthesis Efficiency: Leveraging Large Language Models and Agentic Workflows for Optimized Literature Screening.Cochrane evidence synthesis and methods · 2025Article
- Comparison of Elicit AI and Traditional Literature Searching in Evidence Syntheses Using Four Case Studies.Cochrane evidence synthesis and methods · 2025Article
- Exploring the Role of Artificial Intelligence in Evidence Synthesis: Insights From the CORE Information Retrieval Forum 2025.Cochrane evidence synthesis and methods · 2025Article
- Artificial Intelligence Search Tools for Evidence Synthesis: Comparative Analysis and Implementation Recommendations.Cochrane evidence synthesis and methods · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
introductionWith the increasing accessibility of tools such as ChatGPT, Copilot, DeepSeek, Dall-E, and Gemini, generative artificial intelligence (GenAI) has been poised as a potential, research timesaving tool, especially for synthesising evidence. Our objective was to determine whether GenAI can assist with evidence synthesis by assessing its performance using its accuracy, error rates, and time savings compared to the traditional expert-driven approach.
methodsTo systematically review the evidence, we searched five databases on 17 January 2025, synthesised outcomes reporting on the accuracy, error rates, or time taken, and appraised the risk-of-bias using a modified version of QUADAS-2.
resultsWe identified 3,071 unique records, 19 of which were included in our review. Most studies had a high or unclear risk-of-bias in Domain 1A: review selection, Domain 2A: GenAI conduct, and Domain 1B: applicability of results. When used for (1) searching GenAI missed 68% to 96% (median = 91%) of studies, (2) screening made incorrect inclusion decisions ranging from 0% to 29% (median = 10%); and incorrect exclusion decisions ranging from 1% to 83% (median = 28%), (3) incorrect data extractions ranging from 4% to 31% (median = 14%), (4) incorrect risk-of-bias assessments ranging from 10% to 56% (median = 27%).
conclusionOur review shows that the current evidence does not support GenAI use in evidence synthesis without human involvement or oversight. However, for most tasks other than searching, GenAI may have a role in assisting humans with evidence synthesis.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.