ArticleSystematic reviews2023
Are ChatGPT and large language models "the answer" to bringing us closer to systematic review automation?
Article in Systematic reviews, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 70 papers, 6 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
70 citing papers in PubMed, 6 syntheses or guidelines pooled it.
- A Systematic Review of Genes Affecting Endocochlear Potential.Journal of the Association for Research in Otolaryngology : JARO · 2026Pooled it
- Faster but Not Truer, Part 1: A Systematic Scoping Review of Artificial Intelligence in Health Economics and Outcomes Research.Journal of health economics and outcomes research · 2026Pooled it
- Evaluation of a large language model (ChatGPT) versus human researchers in assessing risk-of-bias and community engagement levels: a systematic review use-case analysis.European journal of public health · 2025Pooled it
- The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review.Journal of the American Medical Informatics Association : JAMIA · 2025Pooled it
- How much can we save by applying artificial intelligence in evidence synthesis? Results from a pragmatic review to quantify workload efficiencies and cost savings.Frontiers in pharmacology · 2025Pooled it
- Evaluation of a prototype machine learning tool to semi-automate data extraction for systematic literature reviews.Systematic reviews · 2023Pooled it
- Developing and Evaluating the Use of ChatGPT as a Screening Tool for Nurses Conducting Structured Literature Reviews: Proof of Concept Study Results.Journal of clinical nursing · 2026Article
- Can artificial intelligence accurately assess systematic review quality? Benchmarking large language models for AMSTAR 2 appraisal in dental evidence synthesis.Evidence-based dentistry · 2026Article
- The Internalized Ableism Inventory: Scale development using a hybrid artificial intelligence and community-based participatory research design.Rehabilitation psychology · 2026Article
- Performance of Zero-Shot Classifiers for Categorizing RCT Abstracts by Intervention Type: Validation Study.JMIR medical informatics · 2026Article
- Evaluating Elicit's systematic reviews workflow in an umbrella review on air pollution and acute lower respiratory infections: a methodological study for quality appraisal.BMC medical research methodology · 2026Article
- Ontology-driven generation of parameters for health technology assessment models: a prompt engineering study.International journal of technology assessment in health care · 2026Article
- Optimizing document retrieval using massive text embeddings and LLM prompt engineering.Systematic reviews · 2026Article
- Clinical research on homeopathic preparations: protocol template for a series of systematic reviews.Systematic reviews · 2026Article
- Rationale and Design of the Cancer Immunotherapy Evidence Living (CIEL) Library: A Continuously Updated Clinical Trial Database of Cancer Immunotherapies.Health science reports · 2026Article
- Applications of Large Language Models in Medical Research: From Systematic Reviews to Clinical Studies.Bioengineering (Basel, Switzerland) · 2026Review
- AI-Assisted Systematic Review: Humans Still Need to Review All Abstracts for Inclusion.JMIR formative research · 2026Article
- AI for scientific discovery is a social problem.Patterns (New York, N.Y.) · 2026Review
- Compact large language models for title and abstract screening in systematic reviews: An assessment of feasibility, accuracy, and workload reduction.Research synthesis methods · 2026Article
- Using Large Language Models to Summarize Evidence in Biomedical Articles: Exploratory Comparison Between AI- and Human-Annotated Bibliographies.JMIR formative research · 2026Article
10 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
In this commentary, we discuss ChatGPT and our perspectives on its utility to systematic reviews (SRs) through the appropriateness and applicability of its responses to SR related prompts. The advancement of artificial intelligence (AI)-assisted technologies leave many wondering about the current capabilities, limitations, and opportunities for integration AI into scientific endeavors. Large language models (LLM)-such as ChatGPT, designed by OpenAI-have recently gained widespread attention with their ability to respond to various prompts in a natural-sounding way. Systematic reviews (SRs) utilize secondary data and often require many months and substantial financial resources to complete, making them attractive grounds for developing AI-assistive technologies. On February 6, 2023, PICO Portal developers hosted a webinar to explore ChatGPT's responses to tasks related to SR methodology. Our experience from exploring the responses of ChatGPT suggest that while ChatGPT and LLMs show some promise for aiding in SR-related tasks, the technology is in its infancy and needs much development for such applications. Furthermore, we advise that great caution should be taken by non-content experts in using these tools due to much of the output appearing, at a high level, to be valid, while much is erroneous and in need of active vetting.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.