Evidence map›Paper›PMID 31805934›Full record

ArticleBMC medical informatics and decision making2019

Improving reference prioritisation with PICO recognition.

Austin J Brockmeier, Meizhi Ju, Piotr Przybyła, Sophia Ananiadou

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in BMC medical informatics and decision making, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers, 4 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
23citing papers in PubMed, 4 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

23 citing papers in PubMed, 4 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Trial
  6. Article
  7. Review
  8. Article
  9. Article
  10. Article
  11. PICOT questions and search strategies formulation: A novel approach using artificial intelligence automation.Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing · 2025
    Article
  12. Article
  13. Article
  14. Making guidelines computable.Clinical and public health guidelines · 2024
    Article
  15. Article
  16. Review
  17. Article
  18. Article
  19. A Scoping Review of Adopted Information Extraction Methods for RCTs.Medical journal of the Islamic Republic of Iran · 2023
    Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors.

Austin J BrockmeierNational Centre of Text Mining, School of Computer Science, University of Manchester, Princess Street, Manchester, M1 7DN, UK.
Meizhi JuNational Centre of Text Mining, School of Computer Science, University of Manchester, Princess Street, Manchester, M1 7DN, UK.
Piotr PrzybyłaNational Centre of Text Mining, School of Computer Science, University of Manchester, Princess Street, Manchester, M1 7DN, UK.
Sophia AnaniadouNational Centre of Text Mining, School of Computer Science, University of Manchester, Princess Street, Manchester, M1 7DN, UK. sophia.ananiadou@manchester.ac.uk.ORCID 0000-0002-4097-9191

Funding

Medical Research Council MR/L01078X/1Medical Research Council MR/N00583X/1
6 · The paper itself

Abstract

backgroundMachine learning can assist with multiple tasks during systematic reviews to facilitate the rapid retrieval of relevant references during screening and to identify and extract information relevant to the study characteristics, which include the PICO elements of patient/population, intervention, comparator, and outcomes. The latter requires techniques for identifying and categorising fragments of text, known as named entity recognition.

methodsA publicly available corpus of PICO annotations on biomedical abstracts is used to train a named entity recognition model, which is implemented as a recurrent neural network. This model is then applied to a separate collection of abstracts for references from systematic reviews within biomedical and health domains. The occurrences of words tagged in the context of specific PICO contexts are used as additional features for a relevancy classification model. Simulations of the machine learning-assisted screening are used to evaluate the work saved by the relevancy model with and without the PICO features. Chi-squared and statistical significance of positive predicted values are used to identify words that are more indicative of relevancy within PICO contexts.

resultsInclusion of PICO features improves the performance metric on 15 of the 20 collections, with substantial gains on certain systematic reviews. Examples of words whose PICO context are more precise can explain this increase.

conclusionsWords within PICO tagged segments in abstracts are predictive features for determining inclusion. Combining PICO annotation model into the relevancy classification pipeline is a promising approach. The annotations may be useful on their own to aid users in pinpointing necessary information for data extraction, or to facilitate semantic search.

Indexed as

Databases, GeneticInformation DisseminationMachine LearningSemanticsSystematic Reviews as TopicHumansActive learningEvidence-based medicineLogistic regressionMachine learningSystematic reviewText mining

Identifiers

PMID31805934
PMCPMC6896258

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.