Evidence map›Paper›PMID 32203803›Full record

ArticleEnvironment international2020

SWIFT-Active Screener: Accelerated document screening through active learning and integrated recall estimation.

Brian E Howard, Jason Phillips, Arpit Tandon, Adyasha Maharana, Rebecca Elmore, Deepak Mav, Alex Sedykh, Kristina Thayer, B Alex Merrick, Vickie Walker and 2 more

Open access · goldAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Environment international, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 80 papers, 14 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
80citing papers in PubMed, 14 pooled it
11.5field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

80 citing papers in PubMed, 14 syntheses or guidelines pooled it, 149 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Characterizing the near-road NOEnvironmental pollution and management (Amsterdam, Netherlands) · 2026
    Article
  16. Review
  17. Review
  18. Review
  19. Article
  20. Review

20 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

12 authors at 2 institutions in 1 country.

Brian E HowardSciome LLC, 2 Davis Drive Durham, NC 27709, USA. Electronic address: brian.howard@sciome.com.
Jason PhillipsSciome LLC, 2 Davis Drive Durham, NC 27709, USA.
Arpit TandonSciome LLC, 2 Davis Drive Durham, NC 27709, USA.
Adyasha MaharanaSciome LLC, 2 Davis Drive Durham, NC 27709, USA.
Rebecca ElmoreSciome LLC, 2 Davis Drive Durham, NC 27709, USA.
Deepak MavSciome LLC, 2 Davis Drive Durham, NC 27709, USA.
Alex SedykhSciome LLC, 2 Davis Drive Durham, NC 27709, USA.
Kristina ThayerIntegrated Risk Information System (IRIS) Division, Environmental Protection Agency, 109 T.W. Alexander Drive RTP, NC 27709, USA.
B Alex MerrickNational Toxicology Program (NTP)/National Institute of Environmental Health Sciences (NIEHS), 111 T.W. Alexander Drive RTP, NC 27709, USA.
Vickie WalkerNational Toxicology Program (NTP)/National Institute of Environmental Health Sciences (NIEHS), 111 T.W. Alexander Drive RTP, NC 27709, USA.
Andrew RooneyNational Toxicology Program (NTP)/National Institute of Environmental Health Sciences (NIEHS), 111 T.W. Alexander Drive RTP, NC 27709, USA.
Ruchir R ShahSciome LLC, 2 Davis Drive Durham, NC 27709, USA.
National Institute of Environmental Health Sciences · USEnvironmental Protection Agency · US

Funding

SWIFT-ActiveScreener: research and development of an intelligent web-based document screening systemR43ES029001 · NIEHS · SCIOME, LLC · PI HOWARD, BRIAN · 2017 to 2017
$212k
NIEHS NIH HHS R43 ES029001
6 · The paper itself

Abstract

backgroundIn the screening phase of systematic review, researchers use detailed inclusion/exclusion criteria to decide whether each article in a set of candidate articles is relevant to the research question under consideration. A typical review may require screening thousands or tens of thousands of articles in and can utilize hundreds of person-hours of labor.

methodsHere we introduce SWIFT-Active Screener, a web-based, collaborative systematic review software application, designed to reduce the overall screening burden required during this resource-intensive phase of the review process. To prioritize articles for review, SWIFT-Active Screener uses active learning, a type of machine learning that incorporates user feedback during screening. Meanwhile, a negative binomial model is employed to estimate the number of relevant articles remaining in the unscreened document list. Using a simulation involving 26 diverse systematic review datasets that were previously screened by reviewers, we evaluated both the document prioritization and recall estimation methods.

resultsOn average, 95% of the relevant articles were identified after screening only 40% of the total reference list. In the 5 document sets with 5,000 or more references, 95% recall was achieved after screening only 34% of the available references, on average. Furthermore, the recall estimator we have proposed provides a useful, conservative estimate of the percentage of relevant documents identified during the screening process.

conclusionSWIFT-Active Screener can result in significant time savings compared to traditional screening and the savings are increased for larger project sizes. Moreover, the integration of explicit recall estimation during screening solves an important challenge faced by all machine learning systems for document screening: when to stop screening a prioritized reference list. The software is currently available in the form of a multi-user, collaborative, online web application.

Indexed as

Machine LearningResearchSoftwareActive learningDocument screeningEvidence mappingMachine learningRecall estimationSystematic review

Identifiers

PMID32203803
PMCPMC8082972
OpenAlexW3010940523

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.