ArticleJournal of healthcare informatics research2024
Identifying and Extracting Rare Diseases and Their Phenotypes with Large Language Models.
Article in Journal of healthcare informatics research, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 29 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
29 citing papers in PubMed.
- Large language models and conditional rules in clinical decision support systems.Health information science and systems · 2026Article
- Persona-Driven Data Augmentation for Disease Name Recognition Across Rare and General Disease Corpora: Comparative Evaluation Study.JMIR medical informatics · 2026Article
- Adaptive Fast-Slow Large Language Model Framework for Multidimensional Classification of Prenatal Ultrasound Reports: Comparative Study.Journal of medical Internet research · 2026Article
- An Entity-Based Visual Analytics System Enhancing Medical Expertise Acquisition: Development and Verification Study.JMIR medical informatics · 2026Article
- Synonym Augmentation for Rare Disease Identification in Unstructured Data.medRxiv : the preprint server for health sciences · 2026Article
- Open-Source Large Language Models Distilled DeepSeek-R1 Pose Challenges for On-Premises Clinical Deployment in Medical Diagnosis: A Comparative Study of Performance.Journal of medical systems · 2026Article
- Reframing AI for Rare Disease Recognition.Research square · 2026Article
- GEN-KnowRD: Reframing AI for Rare Disease Recognition.medRxiv : the preprint server for health sciences · 2026Article
- ChatGPT in precision medicine.APL bioengineering · 2026Review
- Leveraging large language models for rare disease named entity recognition.PLOS digital health · 2026Article
- Article
- Inference Gap in Domain Expertise and Machine Intelligence in Named Entity Recognition: Creation of and Insights from a Substance Use-related Dataset.Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing · 2026Article
- Evaluating the Accuracy of the Frysian Questionnaire for Differentiation of Musculoskeletal Complaints for Triage of Musculoskeletal Diseases: Algorithm Development and Validation Study.JMIR medical informatics · 2025Article
- From Conversation to Standardized Terminology: An LLM-RAG Approach for Automated Health Problem Identification in Home Healthcare.Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing · 2025Article
- Prompt Engineering in Clinical Practice: Tutorial for Clinicians.Journal of medical Internet research · 2025Article
- Article
- Large Language Model Architectures in Health Care: Scoping Review of Research Perspectives.Journal of medical Internet research · 2025Article
- Unleashing the potential of prompt engineering for large language models.Patterns (New York, N.Y.) · 2025Review
- Applying artificial intelligence to rare diseases: a literature review highlighting lessons from Fabry disease.Orphanet journal of rare diseases · 2025Review
- Digital evolution: Novo Nordisk's shift to ontology-based data management.Journal of biomedical semantics · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
Purpose: Phenotyping is critical for informing rare disease diagnosis and treatment, but disease phenotypes are often embedded in unstructured text. While natural language processing (NLP) can automate extraction, a major bottleneck is developing annotated corpora. Recently, prompt learning with large language models (LLMs) has been shown to lead to generalizable results without any (zero-shot) or few annotated samples (few-shot), but none have explored this for rare diseases. Our work is the first to study prompt learning for identifying and extracting rare disease phenotypes in the zero- and few-shot settings. Methods: We compared the performance of prompt learning with ChatGPT and fine-tuning with BioClinicalBERT. We engineered novel prompts for ChatGPT to identify and extract rare diseases and their phenotypes (e.g., diseases, symptoms, and signs), established a benchmark for evaluating its performance, and conducted an in-depth error analysis. Results: Overall, fine-tuning BioClinicalBERT resulted in higher performance (F1 of 0.689) than ChatGPT (F1 of 0.472 and 0.610 in the zero- and few-shot settings, respectively). However, ChatGPT achieved higher accuracy for rare diseases and signs in the one-shot setting (F1 of 0.778 and 0.725). Conversational, sentence-based prompts generally achieved higher accuracy than structured lists. Conclusion: Prompt learning using ChatGPT has the potential to match or outperform fine-tuning BioClinicalBERT at extracting rare diseases and signs with just one annotated sample. Given its accessibility, ChatGPT could be leveraged to extract these entities without relying on a large, annotated corpus. While LLMs can support rare disease phenotyping, researchers should critically evaluate model outputs to ensure phenotyping accuracy.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.