Evidence map›Paper›PMID 29778096›Full record

ArticleSystematic reviews2018

Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR).

Elaine Beller, Justin Clark, Guy Tsafnat, Clive Adams, Heinz Diehl, Hans Lund, Mourad Ouzzani, Kristina Thayer, James Thomas, Tari Turner and 4 more

Abstract readLetter
PubMed Full text at PMC Publisher
In one paragraph

Article in Systematic reviews, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 57 papers, 6 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
57citing papers in PubMed, 6 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

57 citing papers in PubMed, 6 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Review
  8. Article
  9. Article
  10. Article
  11. Review
  12. Article
  13. Article
  14. Article
  15. Article
  16. New opportunities and challenges for conservation evidence synthesis from advances in natural language processing.Conservation biology : the journal of the Society for Conservation Biology · 2025
    Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

14 authors.

Elaine BellerCentre for Research in Evidence-Based Practice, Bond University, Robina, Australia. ebeller@bond.edu.au.ORCID 0000-0002-3241-2611
Justin ClarkCentre for Research in Evidence-Based Practice, Bond University, Robina, Australia.
Guy TsafnatAustralian Institute of Health Innovation, Macquarie University, Sydney, Australia.
Clive AdamsFaculty of Medicine and Health Sciences, University of Nottingham, Nottingham, UK.
Heinz DiehlCentre for Evidence-Based Practice, Bergen University College, Bergen, Norway.
Hans LundWestern Norway University of Applied Sciences, Bergen, Norway.
Mourad OuzzaniQatar Computing Research Institute, Hamad Bin Khalifa University, Doha, Qatar.
Kristina ThayerNational Institute of Environmental Health Sciences, PennState University, Pennsylvania, USA.
James ThomasUniversity College London, London, UK.
Tari TurnerMonash University, Clayton, Australia.
Jun XiaFaculty of Medicine and Health Sciences, University of Nottingham, Nottingham, UK.
Karen RobinsonJHU Evidence-based Practice Center, Johns Hopkins University, Baltimore, USA.
Paul GlasziouCentre for Research in Evidence-Based Practice, Bond University, Robina, Australia.
founding members of the ICASR group

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Systematic reviews (SR) are vital to health care, but have become complicated and time-consuming, due to the rapid expansion of evidence to be synthesised. Fortunately, many tasks of systematic reviews have the potential to be automated or may be assisted by automation. Recent advances in natural language processing, text mining and machine learning have produced new algorithms that can accurately mimic human endeavour in systematic review activity, faster and more cheaply. Automation tools need to be able to work together, to exchange data and results. Therefore, we initiated the International Collaboration for the Automation of Systematic Reviews (ICASR), to successfully put all the parts of automation of systematic review production together. The first meeting was held in Vienna in October 2015. We established a set of principles to enable tools to be developed and integrated into toolkits.This paper sets out the principles devised at that meeting, which cover the need for improvement in efficiency of SR tasks, automation across the spectrum of SR tasks, continuous improvement, adherence to high quality standards, flexibility of use and combining components, the need for a collaboration and varied skills, the desire for open source, shared code and evaluation, and a requirement for replicability through rigorous and open evaluation.Automation has a great potential to improve the speed of systematic reviews. Considerable work is already being done on many of the steps involved in a review. The 'Vienna Principles' set out in this paper aim to guide a more coordinated effort which will allow the integration of work by separate teams and build on the experience, code and evaluations done by the many teams working across the globe.

Indexed as

Data AccuracySystematic Reviews as TopicAlgorithmsAutomationCooperative BehaviorData MiningHumansMachine LearningNatural Language ProcessingAutomationCollaborationSystematic review

Identifiers

PMID29778096
PMCPMC5960503

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.