ArticleDatabase : the journal of biological databases and curation2019
Large expert-curated database for benchmarking document similarity detection in biomedical literature search.
Article in Database : the journal of biological databases and curation, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 21 papers, 4 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
21 citing papers in PubMed, 4 syntheses or guidelines pooled it.
- Educational interventions and communication strategies to improve HPV immunization uptake: a systematic literature review.Frontiers in public health · 2025Pooled it
- Examining the Role of Community Engagement in Enhancing the Participation of Racial and Ethnic Minoritized Communities in Alzheimer's Disease Clinical Trials; A Rapid Review.The journal of prevention of Alzheimer's disease · 2024Pooled it
- Epidemiological Impact of Metabolic Syndrome in Overweight and Obese European Children and Adolescents: A Systematic Literature Review.Nutrients · 2023Pooled it
- Peristomal Skin Complications in Ileostomy and Colostomy Patients: What We Need to Know from a Public Health Perspective.International journal of environmental research and public health · 2022Pooled it
- Benchmarking MeSH-augmented embeddings for biomedical document similarity.Journal of biomedical semantics · 2026Article
- Lifetime chemical sensor arrays of organic fluorophores for bacterial fingerprinting.Nature communications · 2026Article
- Silent Burden of Urinary Tract Infections in Intermittent Catheter Users with Neurological Disorders: A Scoping Review.Diseases (Basel, Switzerland) · 2026Review
- Integrating curation into scientific publishing to train AI models.Bioinformatics (Oxford, England) · 2026Article
- VectorSage: enhancing PubMed article retrieval with advanced semantic search.Bioinformatics advances · 2026Article
- LitSense 2.0: AI-powered biomedical information retrieval with sentence and passage level knowledge discovery.Nucleic acids research · 2025Article
- The Adverse Effects and Use of Bevacizumab in Patients with Glioblastoma: A Systematic Review and Meta-Analysis.Pharmaceuticals (Basel, Switzerland) · 2025Review
- BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers.Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing · 2024Article
- MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval.Bioinformatics (Oxford, England) · 2023Article
- Article
- Semantic Annotation of Experimental Methods in Analytical Chemistry.Analytical chemistry · 2022Article
- [Health Technology Assessment of the Probiotic Cleaning Hygiene System (PCHS)].Journal of preventive medicine and hygiene · 2022Article
- Realization of Intelligent Scoring System of Taekwondo Protective Gear under the Application of Neural Network BP Model.Computational intelligence and neuroscience · 2022Article
- Stabilometric Biofeedback Training in Cognitive and Affective Function Improvement. Contribution of the Russian Scientific School. Part II.Human physiology · 2022Article
- Psychophysiological Indicators of Postural Control. Contribution of the Russian Scientific School. Part I.Human physiology · 2022Article
- Perspective: Towards Automated Tracking of Content and Evidence Appraisal of Nutrition Research.Advances in nutrition (Bethesda, Md.) · 2020Article
Corrections and comments
- Erratum issued
Authors and funding
3 authors.
Funding
Abstract
Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.