Observational studyJournal of epidemiology2024
Efficiency and Workload Reduction of Semi-automated Citation Screening Software for Creating Clinical Practice Guidelines: A Prospective Observational Study.
Observational study in Journal of epidemiology, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
8 citing papers in PubMed.
- Artificial Intelligence Resources for the Screening of Titles and Abstracts in Systematic Reviews: A Scoping Review.Cochrane evidence synthesis and methods · 2026Review
- Artificial intelligence-driven study selection in systematic reviews of randomized controlled trials, emulated trials and economic evaluation studies using large language models.PLOS digital health · 2026Article
- Benchmarking and efficiency assessment in intensive care units: a systematic review protocol.Critical care science · 2026Article
- Performance of large language models and prompt engineering strategies for data extraction in systematic reviews.Frontiers in digital health · 2026Article
- Optimal large language models to screen citations for systematic reviews.Research synthesis methods · 2025Article
- Accuracy of Large Language Models for Literature Screening in Thoracic Surgery: Diagnostic Study.Journal of medical Internet research · 2025Article
- The dynamic linkage between covid-19 and nutrition: a review from a probiotics perspective using machine learning and bibliometric analysis.Frontiers in nutrition · 2025Article
- Performance of a Large Language Model in Screening Citations.JAMA network open · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundWe evaluated the applicability of automated citation screening in developing clinical practice guidelines.
methodsWe prospectively compared the efficiency of citation screening between the conventional (Rayyan) and semi-automated (ASReview software) methods. We searched the literature for five clinical questions (CQs) in the development of the Japanese Clinical Practice Guidelines for the Management of Sepsis and Septic Shock. Objective measurements of the time required to complete citation screening were recorded. Following the first screening round, in the primary analysis, the sensitivity, specificity, positive predictive value, and overall screening time were calculated for both procedures using the semi-automated tool as index and the results of the conventional method as standard reference. In the secondary analysis, the same parameters were compared between the two procedures using the final list of included studies after the second screening session as standard reference.
resultsAmong the five CQs after the first screening session, the highest and lowest sensitivity, specificity, and positive predictive values were 0.241 and 0.795; 0.991 and 1.000; and 0.482 and 0.929, respectively. In the secondary analysis, the highest sensitivity and specificity in the semi-automated citation screening were 1.000 and 0.997, respectively. The overall screening time per 100 studies was significantly shorter with semi-automated than with conventional citation screening.
conclusionThe potential advantages of the semi-automated method (shorter screening time and higher discriminatory rate for the final list of studies) warrant further validation.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.