Evidence map›Paper›PMID 26073974›Full record

ArticleSystematic reviews2015

Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers.

John Rathbone, Tammy Hoffmann, Paul Glasziou

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Systematic reviews, 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 94 papers, 27 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
94citing papers in PubMed, 27 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

94 citing papers in PubMed, 27 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Emergent effects of global change on consumption depend on consumers and their resources in marine systems.Proceedings of the National Academy of Sciences of the United States of America · 2022
    Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

34 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

John RathboneCentre for Research in Evidence-Based Practice, Bond University, Gold Coast, Australia. jrathbon@bond.edu.au.
Tammy HoffmannCentre for Research in Evidence-Based Practice, Bond University, Gold Coast, Australia. thoffmann@bond.edu.au.
Paul GlasziouCentre for Research in Evidence-Based Practice, Bond University, Gold Coast, Australia. pglaszio@bond.edu.au.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundCitation screening is time consuming and inefficient. We sought to evaluate the performance of Abstrackr, a semi-automated online tool for predictive title and abstract screening.

methodsFour systematic reviews (aHUS, dietary fibre, ECHO, rituximab) were used to evaluate Abstrackr. Citations from electronic searches of biomedical databases were imported into Abstrackr, and titles and abstracts were screened and included or excluded according to the entry criteria. This process was continued until Abstrackr predicted and classified the remaining unscreened citations as relevant or irrelevant. These classification predictions were checked for accuracy against the original review decisions. Sensitivity analyses were performed to assess the effects of including case reports in the aHUS dataset whilst screening and the effects of using larger imbalanced datasets with the ECHO dataset. The performance of Abstrackr was calculated according to the number of relevant studies missed, the workload saving, the false negative rate, and the precision of the algorithm to correctly predict relevant studies for inclusion, i.e. further full text inspection.

resultsOf the unscreened citations, Abstrackr's prediction algorithm correctly identified all relevant citations for the rituximab and dietary fibre reviews. However, one relevant citation in both the aHUS and ECHO reviews was incorrectly predicted as not relevant. The workload saving achieved with Abstrackr varied depending on the complexity and size of the reviews (9 % rituximab, 40 % dietary fibre, 67 % aHUS, and 57 % ECHO). The proportion of citations predicted as relevant, and therefore, warranting further full text inspection (i.e. the precision of the prediction) ranged from 16 % (aHUS) to 45 % (rituximab) and was affected by the complexity of the reviews. The false negative rate ranged from 2.4 to 21.7 %. Sensitivity analysis performed on the aHUS dataset increased the precision from 16 to 25 % and increased the workload saving by 10 % but increased the number of relevant studies missed. Sensitivity analysis performed with the larger ECHO dataset increased the workload saving (80 %) but reduced the precision (6.8 %) and increased the number of missed citations.

conclusionsSemi-automated title and abstract screening with Abstrackr has the potential to save time and reduce research waste.

Indexed as

Abstracting and IndexingAlgorithmsReview Literature as TopicHumansReproducibility of ResultsSensitivity and Specificity

Identifiers

PMID26073974
PMCPMC4472176

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.