ArticleJournal of the American Medical Informatics Association : JAMIA2024
Fine-tuning large language models for rare disease concept normalization.
Article in Journal of the American Medical Informatics Association : JAMIA, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 32 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
32 citing papers in PubMed.
- The Rise of Small Language Models in Healthcare: A Comprehensive Survey.Computer science review · 2026Article
- Symptom Terminology Normalization in Traditional Chinese Medicine: Development and Evaluation of a 2-Stage Deep Learning Framework Based on Fine-Grained Semantic Classification.JMIR medical informatics · 2026Article
- Large language models are powerful electronic health record encoders.NPJ digital medicine · 2026Article
- Synonym Augmentation for Rare Disease Identification in Unstructured Data.medRxiv : the preprint server for health sciences · 2026Article
- Integrative machine learning and molecular docking reveal Narirutin targets ABCC1/ABCG2 for immune microenvironment remodeling in colorectal cancer.Scientific reports · 2026Article
- Article
- Digital twin-supported behavioral intention in mothers of young children to prevent childhood obesity: a large language model-based intervention study.Frontiers in artificial intelligence · 2026Article
- From Zero-Shot to Bedside: A Practical Playbook for Adapting Open-Source Large Language Models to Clinical Symptom Extraction.Proceedings of machine learning research · 2026Article
- From memorization to generalization: fine-tuning large language models for biomedical term-to-identifier normalization.Frontiers in digital health · 2026Article
- Large Language Models in Bio-Ontology Research: A Review.Bioengineering (Basel, Switzerland) · 2025Review
- An integrated approach for rare disease detection and classification in Spanish pediatric medical reports.Scientific reports · 2025Article
- Automated Esophageal Cancer Staging From Free-Text Radiology Reports: Large Language Model Evaluation Study.JMIR medical informatics · 2025Article
- Large Language Model-Enhanced Drug Repositioning Knowledge Extraction via Long Chain-of-Thought: Development and Evaluation Study.JMIR medical informatics · 2025Article
- Ontology accelerates few-shot learning capability of large language model: A study in extraction of drug efficacy in a rare pediatric epilepsy.International journal of medical informatics · 2025Article
- Improving automated deep phenotyping through large language models using retrieval-augmented generation.Genome medicine · 2025Article
- The Development Landscape of Large Language Models for Biomedical Applications.Annual review of biomedical data science · 2025Review
- Evaluating the role of large language models in traditional Chinese medicine diagnosis and treatment recommendations.NPJ digital medicine · 2025Article
- Biomedical text normalization through generative modeling.Journal of biomedical informatics · 2025Article
- CDEMapper: enhancing National Institutes of Health common data element use with large language models.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Article
Corrections and comments
- Update of
Authors and funding
4 authors.
Funding
Abstract
objectiveWe aim to develop a novel method for rare disease concept normalization by fine-tuning Llama 2, an open-source large language model (LLM), using a domain-specific corpus sourced from the Human Phenotype Ontology (HPO).
methodsWe developed an in-house template-based script to generate two corpora for fine-tuning. The first (NAME) contains standardized HPO names, sourced from the HPO vocabularies, along with their corresponding identifiers. The second (NAME+SYN) includes HPO names and half of the concept's synonyms as well as identifiers. Subsequently, we fine-tuned Llama 2 (Llama2-7B) for each sentence set and conducted an evaluation using a range of sentence prompts and various phenotype terms.
resultsWhen the phenotype terms for normalization were included in the fine-tuning corpora, both models demonstrated nearly perfect performance, averaging over 99% accuracy. In comparison, ChatGPT-3.5 has only ∼20% accuracy in identifying HPO IDs for phenotype terms. When single-character typos were introduced in the phenotype terms, the accuracy of NAME and NAME+SYN is 10.2% and 36.1%, respectively, but increases to 61.8% (NAME+SYN) with additional typo-specific fine-tuning. For terms sourced from HPO vocabularies as unseen synonyms, the NAME model achieved 11.2% accuracy, while the NAME+SYN model achieved 92.7% accuracy.
conclusionOur fine-tuned models demonstrate ability to normalize phenotype terms unseen in the fine-tuning corpus, including misspellings, synonyms, terms from other ontologies, and laymen's terms. Our approach provides a solution for the use of LLMs to identify named medical entities from clinical narratives, while successfully normalizing them to standard concepts in a controlled vocabulary.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.