ArticleJournal of biomedical semantics2017
12 years on - Is the NLM medical text indexer still useful and relevant?
Article in Journal of biomedical semantics, 2017. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 30 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
30 citing papers in PubMed.
- Automated indexing in MEDLINE and the Medical Text Indexer (MTI), 2000-2025: a scoping review.Journal of the Medical Library Association : JMLA · 2026Article
- Where Evidence-Based Medicine Meets AI: Promise, Pitfalls, and Practice.AEM education and training · 2026Article
- Sometimes the apple does fall far from the tree: a case study on automatic indexing precision errors in PubMed.Journal of the Medical Library Association : JMLA · 2025Article
- Enhancing automated indexing of publication types and study designs in biomedical literature using full-text features.medRxiv : the preprint server for health sciences · 2025Article
- Publication Type Tagging using Transformer Models and Multi-Label Classification.medRxiv : the preprint server for health sciences · 2025Article
- Algorithmic indexing in MEDLINE frequently overlooks important concepts and may compromise literature search results.Journal of the Medical Library Association : JMLA · 2025Article
- Constructing a finer-grained representation of clinical trial results from ClinicalTrials.gov.Scientific data · 2024Article
- A Bibliographic Dataset of Health Artificial Intelligence Research.Health data science · 2024Article
- Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?PloS one · 2024Article
- Publication Type Tagging using Transformer Models and Multi-Label Classification.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2024Article
- Automated indexing using NLM's Medical Text Indexer (MTI) compared to human indexing in Medline: a pilot study.Journal of the Medical Library Association : JMLA · 2023Article
- The road from manual to automatic semantic indexing of biomedical literature: a 10 years journey.Frontiers in research metrics and analytics · 2023Article
- Multi-probe attention neural network for COVID-19 semantic indexing.BMC bioinformatics · 2022Article
- GenomicSuperSignature facilitates interpretation of RNA-seq experiments through robust, efficient comparison to public databases.Nature communications · 2022Article
- An automated multi-modal graph-based pipeline for mouse genetic discovery.Bioinformatics (Oxford, England) · 2022Article
- Artificial intelligence in clinical and translational science: Successes, challenges and opportunities.Clinical and translational science · 2022Review
- Preliminary comparison of the performance of the National Library of Medicine's systematic review publication type and the sensitive clinical queries filter for systematic reviews in PubMed.Journal of the Medical Library Association : JMLA · 2022Article
- Dr. Lindberg and scholarly publishing.Information services & use · 2022Article
- Identification of most influential co-occurring gene suites for gastrointestinal cancer using biomedical literature mining and graph-based influence maximization.BMC medical informatics and decision making · 2020Article
- FullMeSH: improving large-scale MeSH indexing with full text.Bioinformatics (Oxford, England) · 2020Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
Abstract
backgroundFacing a growing workload and dwindling resources, the US National Library of Medicine (NLM) created the Indexing Initiative project in 1996. This cross-library team's mission is to explore indexing methodologies for ensuring quality and currency of NLM document collections. The NLM Medical Text Indexer (MTI) is the main product of this project and has been providing automated indexing recommendations since 2002. After all of this time, the questions arise whether MTI is still useful and relevant.
methodsTo answer the question about MTI usefulness, we track a wide variety of statistics related to how frequently MEDLINE indexers refer to MTI recommendations, how well MTI performs against human indexing, and how often MTI is used. To answer the question of MTI relevancy compared to other available tools, we have participated in the 2013 and 2014 BioASQ Challenges. The BioASQ Challenges have provided us with an unbiased comparison between the MTI system and other systems performing the same task.
resultsIndexers have continually increased their use of MTI recommendations over the years from 15.75% of the articles they index in 2002 to 62.44% in 2014 showing that the indexers find MTI to be increasingly useful. The MTI performance statistics show significant improvement in Precision (+0.2992) and F
conclusionsBased on our findings, yes, MTI is still relevant and useful, and needs to be improved and expanded. The BioASQ Challenge results have shown that we need to incorporate more machine learning into MTI while still retaining the indexing rules that have earned MTI the indexers' trust over the years. We also need to expand MTI through the use of full text, when and where it is available, to provide coverage of indexing terms that are typically only found in the full text. The role of MTI at NLM is also expanding into new areas, further reinforcing the idea that MTI is increasingly useful and relevant.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.