Evidence map›Paper›PMID 30102703›Full record

ArticlePLoS computational biology2018

Scaling up data curation using deep learning: An application to literature triage in genomic variation resources.

Kyubum Lee, Maria Livia Famiglietti, Aoife McMahon, Chih-Hsuan Wei, Jacqueline Ann Langdon MacArthur, Sylvain Poux, Lionel Breuza, Alan Bridge, Fiona Cunningham, Ioannis Xenarios and 1 more

Abstract read
In one paragraph

Article in PLoS computational biology, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 26 papers.

0numbers the graph read from it
0cells of the map it votes in
26citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

26 citing papers in PubMed.

  1. Review
  2. Article
  3. Article
  4. Article
  5. Review
  6. Article
  7. Article
  8. Deep learning in bioinformatics.Turkish journal of biology = Turk biyoloji dergisi · 2023
    Article
  9. Assigning species information to corresponding genes by a sequence labeling framework.Database : the journal of biological databases and curation · 2022
    Article
  10. A roadmap for the functional annotation of protein families: a community perspective.Database : the journal of biological databases and curation · 2022
    Article
  11. Article
  12. Review
  13. Article
  14. Article
  15. Review
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

11 authors.

Kyubum LeeNational Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, Maryland, United States of America.ORCID 0000-0003-2015-3939
Maria Livia FamigliettiSwiss-Prot Group, SIB Swiss Institute of Bioinformatics, Geneva, Switzerland.
Aoife McMahonEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, Cambridge, United Kingdom.ORCID 0000-0003-0978-0309
Chih-Hsuan WeiNational Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, Maryland, United States of America.
Jacqueline Ann Langdon MacArthurEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, Cambridge, United Kingdom.
Sylvain PouxSwiss-Prot Group, SIB Swiss Institute of Bioinformatics, Geneva, Switzerland.
Lionel BreuzaSwiss-Prot Group, SIB Swiss Institute of Bioinformatics, Geneva, Switzerland.
Alan BridgeSwiss-Prot Group, SIB Swiss Institute of Bioinformatics, Geneva, Switzerland.ORCID 0000-0003-2148-9135
Fiona CunninghamEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, Cambridge, United Kingdom.
Ioannis XenariosCenter for Integrative Genomics, University of Lausanne, Lausanne Switzerland.
Zhiyong LuNational Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, Maryland, United States of America.ORCID 0000-0001-9998-916X

Funding

National Biomedical Information ServicesZIHLM200888 · NLM · NATIONAL LIBRARY OF MEDICINE · PI PALTOO, DINA · 2009 to 2025
$5200.7M
UniProt: A centralized protein sequence and function resourceU41HG007822 · NHGRI · EUROPEAN MOLECULAR BIOLOGY LABORATORY · PI BATEMAN, ALEX · 2014 to 2017
$21.9M
Named Entity Recognition and Relationship Extraction in BiomedicineZIALM091813 · NLM · NATIONAL LIBRARY OF MEDICINE · PI LU, ZHIYONG · 2013 to 2022
$13.4M
GWAS CatalogU41HG007823 · NHGRI · EUROPEAN MOLECULAR BIOLOGY LABORATORY · PI CUNNINGHAM, FIONA, PARKINSON, HELEN ELIZABETH · 2014 to 2021
$6.7M
NHGRI NIH HHS U41 HG007822NHGRI NIH HHS U41 HG007823
6 · The paper itself

Abstract

Manually curating biomedical knowledge from publications is necessary to build a knowledge based service that provides highly precise and organized information to users. The process of retrieving relevant publications for curation, which is also known as document triage, is usually carried out by querying and reading articles in PubMed. However, this query-based method often obtains unsatisfactory precision and recall on the retrieved results, and it is difficult to manually generate optimal queries. To address this, we propose a machine-learning assisted triage method. We collect previously curated publications from two databases UniProtKB/Swiss-Prot and the NHGRI-EBI GWAS Catalog, and used them as a gold-standard dataset for training deep learning models based on convolutional neural networks. We then use the trained models to classify and rank new publications for curation. For evaluation, we apply our method to the real-world manual curation process of UniProtKB/Swiss-Prot and the GWAS Catalog. We demonstrate that our machine-assisted triage method outperforms the current query-based triage methods, improves efficiency, and enriches curated content. Our method achieves a precision 1.81 and 2.99 times higher than that obtained by the current query-based triage methods of UniProtKB/Swiss-Prot and the GWAS Catalog, respectively, without compromising recall. In fact, our method retrieves many additional relevant publications that the query-based method of UniProtKB/Swiss-Prot could not find. As these results show, our machine learning-based method can make the triage process more efficient and is being implemented in production so that human curators can focus on more challenging tasks to improve the quality of knowledge bases.

Indexed as

Databases, GeneticDatabases, ProteinData CurationDeep LearningGenomicsInformation Storage and RetrievalKnowledge BasesMachine LearningPublications

Identifiers

PMID30102703
PMCPMC6107285

What OpenQuestion holds

Textmetadata
LicenceCC0
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.