Evidence map›Paper›PMID 39114977›Full record

ArticleDatabase : the journal of biological databases and curation2024

The overview of the BioRED (Biomedical Relation Extraction Dataset) track at BioCreative VIII.

Rezarta Islamaj, Po-Ting Lai, Chih-Hsuan Wei, Ling Luo, Tiago Almeida, Richard A A Jonker, Sofia I R Conceição, Diana F Sousa, Cong-Phuoc Phan, Jung-Hsien Chiang and 13 more

Erratum issuedAbstract read
In one paragraph

Article in Database : the journal of biological databases and curation, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 10 papers.

0numbers the graph read from it
0cells of the map it votes in
10citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

10 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. DiMB-RE: mining the scientific literature for diet-microbiome associations.Journal of the American Medical Informatics Association : JAMIA · 2025
    Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

23 authors.

Rezarta IslamajNational Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, United States.ORCID 0000-0001-5651-1860
Po-Ting LaiNational Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, United States.
Chih-Hsuan WeiNational Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, United States.ORCID 0000-0001-5094-7321
Ling LuoSchool of Computer Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China.ORCID 0000-0002-5141-0259
Tiago AlmeidaDepartment of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, Campus Universitário de Santiago, Aveiro 3810-193, Portugal.ORCID 0000-0002-4258-3350
Richard A A JonkerDepartment of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, Campus Universitário de Santiago, Aveiro 3810-193, Portugal.
Sofia I R ConceiçãoDepartamento de Informática, Faculdade de Ciências da Universidade de Lisboa, Edifício C6 Campo Grande, Lisbon 1749-016, Portugal.
Diana F SousaDepartamento de Informática, Faculdade de Ciências da Universidade de Lisboa, Edifício C6 Campo Grande, Lisbon 1749-016, Portugal.
Cong-Phuoc PhanDepartment of Computer Science and Information Engineering, National Cheng Kung University, No.1, University Road, Tainan City 701, Taiwan, Republic of China.
Jung-Hsien ChiangDepartment of Computer Science and Information Engineering, National Cheng Kung University, No.1, University Road, Tainan City 701, Taiwan, Republic of China.
Jiru LiSchool of Computer Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China.ORCID 0009-0001-8093-9784
Dinghao PanSchool of Computer Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China.
Wilailack MeesawadDepartment of Computer Science and Information Engineering, National Central University, No. 300, Zhongda Rd., Zhongli District, Taoyuan City 32001, Taiwan, Republic of China.
Richard Tzong-Han TsaiDepartment of Computer Science and Information Engineering, National Central University, No. 300, Zhongda Rd., Zhongli District, Taoyuan City 32001, Taiwan, Republic of China.
M Janina SarolSchool of Information Sciences, University of Illinois at Urbana-Champaign, 614 E. Daniel St, Champaign, IL 61820, United States.ORCID 0000-0002-0991-4429
Gibong HongSchool of Information Sciences, University of Illinois at Urbana-Champaign, 614 E. Daniel St, Champaign, IL 61820, United States.
Airat ValievHigher School of Economics University, 20 Myasnitskaya st, Moscow 101000, Russia.
Elena TutubalinaHigher School of Economics University, 20 Myasnitskaya st, Moscow 101000, Russia.ORCID 0000-0001-7936-0284
Shao-Man LeeMiin Wu School of Computing, National Cheng Kung University, No. 1, University Road, Tainan 701, Taiwan, Republic of China.
Yi-Yu HsuMiin Wu School of Computing, National Cheng Kung University, No. 1, University Road, Tainan 701, Taiwan, Republic of China.ORCID 0000-0003-4286-3181
Mingjie LiSchool of Computing Technologies, RMIT University, 124 La Trobe Street, Melbourne, Victoria 3000, Australia.
Karin VerspoorSchool of Computing Technologies, RMIT University, 124 La Trobe Street, Melbourne, Victoria 3000, Australia.ORCID 0000-0002-8661-1544
Zhiyong LuNational Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, United States.ORCID 0000-0001-9998-916X

Funding

Machine Learning and Natural Language Processing for Biomedical ApplicationsZIALM237241 · NLM · NATIONAL LIBRARY OF MEDICINE · PI LU, ZHIYONG · 2023 to 2025
$12.2M
Fct UI/BD/153730/2022FCT (Foundation for Science and Technology) 2020.05784 DSAIPA/AI/0088/2020 NSTC 112-2321-B-006-008Fundação para a Ciência e a Tecnologia (FCT)Fundamental Research Funds for the Central UniversitiesKazan Federal UniversityLASIGE Computer Science and Engineering Research CentreResearch Unit UIDB/00408/2020 UIDP/00408/2020Russian Science Foundation 23-11-00358the NIH Intramural Research Program, National Library of Medicine
6 · The paper itself

Abstract

The BioRED track at BioCreative VIII calls for a community effort to identify, semantically categorize, and highlight the novelty factor of the relationships between biomedical entities in unstructured text. Relation extraction is crucial for many biomedical natural language processing (NLP) applications, from drug discovery to custom medical solutions. The BioRED track simulates a real-world application of biomedical relationship extraction, and as such, considers multiple biomedical entity types, normalized to their specific corresponding database identifiers, as well as defines relationships between them in the documents. The challenge consisted of two subtasks: (i) in Subtask 1, participants were given the article text and human expert annotated entities, and were asked to extract the relation pairs, identify their semantic type and the novelty factor, and (ii) in Subtask 2, participants were given only the article text, and were asked to build an end-to-end system that could identify and categorize the relationships and their novelty. We received a total of 94 submissions from 14 teams worldwide. The highest F-score performances achieved for the Subtask 1 were: 77.17% for relation pair identification, 58.95% for relation type identification, 59.22% for novelty identification, and 44.55% when evaluating all of the above aspects of the comprehensive relation extraction. The highest F-score performances achieved for the Subtask 2 were: 55.84% for relation pair, 43.03% for relation type, 42.74% for novelty, and 32.75% for comprehensive relation extraction. The entire BioRED track dataset and other challenge materials are available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/ and https://codalab.lisn.upsaclay.fr/competitions/13377 and https://codalab.lisn.upsaclay.fr/competitions/13378. Database URL: https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/https://codalab.lisn.upsaclay.fr/competitions/13377https://codalab.lisn.upsaclay.fr/competitions/13378.

Indexed as

Data MiningNatural Language ProcessingDatabases, FactualHumansSemantics

Identifiers

PMID39114977
PMCPMC11306928

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.