ArticleBMC bioinformatics2015
An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition.
Article in BMC bioinformatics, 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 100 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
100 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Question answering systems for health professionals at the point of care-a systematic review.Journal of the American Medical Informatics Association : JAMIA · 2024Pooled it
- The Rise of Small Language Models in Healthcare: A Comprehensive Survey.Computer science review · 2026Article
- LitBench: A Benchmark for Retrieval-Grounded Multi-Paper Evidence Synthesis in Epilepsy.medRxiv : the preprint server for health sciences · 2026Article
- RadTechBERT: a domain-specific language model for radiological technology evaluated using Japanese national examination-derived cloze questions.Radiological physics and technology · 2026Article
- Large Language Models in Preclinical Spine Research: A Scoping Review and Expert Perspective on Evidence-Aware Experimental Workflows.JOR spine · 2026Article
- EDEL: enhancing dense retrievers for curation of biomedical knowledge bases.Bioinformatics (Oxford, England) · 2026Article
- A critical evaluation of generative query expansion on biomedical literature retrieval.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Article
- Article
- Soft-Community Kernel Rényi Spectrum for Semantic Uncertainty Estimation in Large Language Models.Entropy (Basel, Switzerland) · 2026Article
- A survey on large language models in biology and chemistry.Experimental & molecular medicine · 2026Review
- Large language models for bioinformatics.Quantitative biology (Beijing, China) · 2026Review
- APMSR: an intelligent QA system for synthetic biology empowered by adaptive prompting and multi-source knowledge retrieval.Scientific reports · 2026Article
- Research on plug-and-play correlation enhancement modules in deep multi-label learning.Scientific reports · 2026Article
- TogoMCP: natural language querying of life-science knowledge graphs via schema-guided LLMs and the Model Context Protocol.Database : the journal of biological databases and curation · 2026Article
- MedHopper: an agentic RAG-LLM system for multi-hop biomedical QA.Database : the journal of biological databases and curation · 2026Article
- MedHopQA: a disease-centred multi-hop reasoning benchmark and evaluation framework for LLM-based biomedical question answering.Database : the journal of biological databases and curation · 2026Article
- Beyond text: quantifying the impact of figure-guided retrieval in biomedical RAG systems.Frontiers in artificial intelligence · 2026Article
- BioMamba: Domain-Adaptive Biomedical Language Models.Health data science · 2026Article
- Article
40 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
22 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThis article provides an overview of the first BIOASQ challenge, a competition on large-scale biomedical semantic indexing and question answering (QA), which took place between March and September 2013. BIOASQ assesses the ability of systems to semantically index very large numbers of biomedical scientific articles, and to return concise and user-understandable answers to given natural language questions by combining information from biomedical articles and ontologies.
resultsThe 2013 BIOASQ competition comprised two tasks, Task 1a and Task 1b. In Task 1a participants were asked to automatically annotate new PUBMED documents with MESH headings. Twelve teams participated in Task 1a, with a total of 46 system runs submitted, and one of the teams performing consistently better than the MTI indexer used by NLM to suggest MESH headings to curators. Task 1b used benchmark datasets containing 29 development and 282 test English questions, along with gold standard (reference) answers, prepared by a team of biomedical experts from around Europe and participants had to automatically produce answers. Three teams participated in Task 1b, with 11 system runs. The BIOASQ infrastructure, including benchmark datasets, evaluation mechanisms, and the results of the participants and baseline methods, is publicly available.
conclusionsA publicly available evaluation infrastructure for biomedical semantic indexing and QA has been developed, which includes benchmark datasets, and can be used to evaluate systems that: assign MESH headings to published articles or to English questions; retrieve relevant RDF triples from ontologies, relevant articles and snippets from PUBMED Central; produce "exact" and paragraph-sized "ideal" answers (summaries). The results of the systems that participated in the 2013 BIOASQ competition are promising. In Task 1a one of the systems performed consistently better from the NLM's MTI indexer. In Task 1b the systems received high scores in the manual evaluation of the "ideal" answers; hence, they produced high quality summaries as answers. Overall, BIOASQ helped obtain a unified view of how techniques from text classification, semantic indexing, document and passage retrieval, question answering, and text summarization can be combined to allow biomedical experts to obtain concise, user-understandable answers to questions reflecting their real information needs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.