ArticleContemporary clinical trials communications2019
The significant cost of systematic reviews and meta-analyses: A call for greater involvement of machine learning to assess the promise of clinical trials.
Article in Contemporary clinical trials communications, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 58 papers, 4 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
58 citing papers in PubMed, 4 syntheses or guidelines pooled it.
- A systematic literature review (SLR) on the adoption of artificial intelligence-assisted SLRS: implications for health technology assessments.International journal of technology assessment in health care · 2026Pooled it
- How much can we save by applying artificial intelligence in evidence synthesis? Results from a pragmatic review to quantify workload efficiencies and cost savings.Frontiers in pharmacology · 2025Pooled it
- Machine learning models for abstract screening task - A systematic literature review application for health economics and outcome research.BMC medical research methodology · 2024Pooled it
- Evaluation of a prototype machine learning tool to semi-automate data extraction for systematic literature reviews.Systematic reviews · 2023Pooled it
- Leveraging prompt-driven generative AI for systematic reviews in digital psychiatry: A stage-matched comparative proof-of-concept for healthcare researchers and clinicians.PLOS digital health · 2026Article
- Can artificial intelligence accurately assess systematic review quality? Benchmarking large language models for AMSTAR 2 appraisal in dental evidence synthesis.Evidence-based dentistry · 2026Article
- Large language models are comparable with commonly used statistical software: A validation of GPT 5.1 for frequentist meta-analysis in orthopaedics.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Article
- AI-Assisted Systematic Literature Review of the Economic Burden of Pneumococcal Disease: Development and Validation Study.JMIR AI · 2026Article
- Using full agreement across multiple large language models for title-and-abstract screening in systematic reviews: a proof-of-concept.Systematic reviews · 2026Article
- Evaluating Elicit's systematic reviews workflow in an umbrella review on air pollution and acute lower respiratory infections: a methodological study for quality appraisal.BMC medical research methodology · 2026Article
- Batch Size Effects on Mid-2025 State-of-the-Art Large Language Model Performance in Automated Title and Abstract Screening.Cochrane evidence synthesis and methods · 2026Article
- Novel Abstract Screening Algorithm Using Delphi-Inspired Large Language Model Consensus for Systematic Reviews in Psychiatry: Nouvel algorithme de sélection des résumés utilisant un consensus issu d'un grand modèle de langage inspiré de la méthode Delphi pour les revues systématiques en psychiatrie.Canadian journal of psychiatry. Revue canadienne de psychiatrie · 2026Article
- Research Waste, Redundancy and the Rise of the Machines: The Questionable Future of Systematic Reviews.Drug and alcohol review · 2026Article
- Assessing Large Language Models for Early Article Identification in Otolaryngology-Head and Neck Surgery Systematic Reviews.Health care science · 2026Article
- Toward an integrated animal-human nutrition intelligence via agentic retrieval-augmented language models.Frontiers in nutrition · 2026Article
- Evaluating AI Performance in Systematic Literature Reviews for HEOR: A Case Study.Journal of health economics and outcomes research · 2026Article
- Streamlining evidence based clinical recommendations with large language models.NPJ digital medicine · 2025Article
- Transparent Reporting of AI in Systematic Literature Reviews: Development of the PRISMA-trAIce Checklist.JMIR AI · 2025Article
- Listening to the Past, Shaping the Future: A Data-mining Based and Visual Analysis of Five Decades of Gastric Carcinogenesis Research.Biological procedures online · 2025Article
- Human-in-the-Loop Artificial Intelligence System for Systematic Literature Review: Methods and Validations for the AutoLit Review Software.Cochrane evidence synthesis and methods · 2025Article
Corrections and comments
- Erratum issued
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundMore than 90% of clinical-trial compounds fail to demonstrate sufficient efficacy and safety. To help alleviate this issue, systematic literature review and meta-analysis (SLR), which synthesize current evidence for a research question, can be applied to preclinical evidence to identify the most promising therapeutics. However, these methods remain time-consuming and labor-intensive. Here, we introduce an economic formula to estimate the expense of SLR for academic institutions and pharmaceutical companies.
methodsWe estimate the manual effort involved in SLR by quantifying the amount of labor required and the total associated labor cost. We begin with an empirical estimation and derive a formula that quantifies and describes the cost.
resultsThe formula estimated that each SLR costs approximately $141,194.80. We found that on average, the ten largest pharmaceutical companies publish 118.71 and the ten major academic institutions publish 132.16 SLRs per year. On average, the total cost of all SLRs per year to each academic institution amounts to $18,660,304.77 and for each pharmaceutical company is $16,761,234.71. DISCUSSION: It appears that SLR is an important, but costly mechanisms to assess the totality of evidence.
conclusionsWith the increase in the number of publications, the significant time and cost of SLR may pose a barrier to their consistent application to assess the promise of clinical trials thoroughly. We call on investigators and developers to develop automated solutions to help with the assessment of preclinical evidence particularly. The formula we introduce provides a cost baseline against which the efficiency of automation can be measured.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.