ArticleJournal of biomedical informatics2024
Model tuning or prompt Tuning? a study of large language models for clinical concept and relation extraction.
Article in Journal of biomedical informatics, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 21 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
21 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Applications of Natural Language Processing and Large Language Models for Social Determinants of Health: Systematic Review.Journal of medical Internet research · 2026Pooled it
- Leveraging Generative Large Language Models for Temporal Relation Extraction From French Clinical Narratives: Prompt-Chaining Approach.Journal of medical Internet research · 2026Article
- A study of large language models for patient information extraction: Model architecture, fine-tuning strategy, and multi-task instruction tuning.Journal of biomedical informatics · 2026Article
- Exploring transformer models: Fine-tuning VS inference on relation extraction from biomedical texts.Computational and structural biotechnology journal · 2026Article
- For clinical data extraction, QLoRA attains accuracy close to LoRA while requiring lower compute resources.medRxiv : the preprint server for health sciences · 2025Article
- Large language models in clinical trials: applications, technical advances, and future directions.BMC medicine · 2025Review
- Enhancing biomedical relation extraction with directionality.Bioinformatics (Oxford, England) · 2025Article
- Semantic Clinical Artificial Intelligence vs Native Large Language Model Performance on the USMLE.JAMA network open · 2025Article
- Decoding substance use disorder severity from clinical notes using a large language model.Npj mental health research · 2025Article
- Advancing Pancreatic Cancer Prediction with a Next Visit Token Prediction Head on Top of Med-BERT.Cancers · 2025Article
- Automatic Summarization of Doctor-Patient Encounter Dialogues Using Large Language Model through Prompt Tuning.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
- Identifying Opioid Overdose and Opioid Use Disorder and Related Information from Clinical Narratives Using Large Language Models.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
- Enhancing Cross-Domain Generalizability in Social Determinants of Health Extraction with Prompt-Tuning Large Language Models.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
- Digital Information Ecosystems in Modern Care Coordination and Patient Care Pathways and the Challenges and Opportunities for AI Solutions.Journal of medical Internet research · 2024Article
- Prompt Engineering Paradigms for Medical Applications: Scoping Review.Journal of medical Internet research · 2024Article
- LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction.Journal of the American Medical Informatics Association : JAMIA · 2024Article
- RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization.Journal of the American Medical Informatics Association : JAMIA · 2024Article
- Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need.Journal of the American Medical Informatics Association : JAMIA · 2024Article
- Large Language Models and Genomics for Summarizing the Role of microRNA in Regulating mRNA Expression.Biomedicines · 2024Article
- Narrative Feature or Structured Feature? A Study of Large Language Models to Identify Cancer Patients at Risk of Heart Failure.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
objectiveTo develop soft prompt-based learning architecture for large language models (LLMs), examine prompt-tuning using frozen/unfrozen LLMs, and assess their abilities in transfer learning and few-shot learning.
methodsWe developed a soft prompt-based learning architecture and compared 4 strategies including (1) fine-tuning without prompts; (2) hard-prompting with unfrozen LLMs; (3) soft-prompting with unfrozen LLMs; and (4) soft-prompting with frozen LLMs. We evaluated GatorTron, a clinical LLM with up to 8.9 billion parameters, and compared GatorTron with 4 existing transformer models for clinical concept and relation extraction on 2 benchmark datasets for adverse drug events and social determinants of health (SDoH). We evaluated the few-shot learning ability and generalizability for cross-institution applications. RESULTS AND
conclusionWhen LLMs are unfrozen, GatorTron-3.9B with soft prompting achieves the best strict F1-scores of 0.9118 and 0.8604 for concept extraction, outperforming the traditional fine-tuning and hard prompt-based models by 0.6 ∼ 3.1 % and 1.2 ∼ 2.9 %, respectively; GatorTron-345 M with soft prompting achieves the best F1-scores of 0.8332 and 0.7488 for end-to-end relation extraction, outperforming other two models by 0.2 ∼ 2 % and 0.6 ∼ 11.7 %, respectively. When LLMs are frozen, small LLMs have a big gap to be competitive with unfrozen models; scaling LLMs up to billions of parameters makes frozen LLMs competitive with unfrozen models. Soft prompting with a frozen GatorTron-8.9B model achieved the best performance for cross-institution evaluation. We demonstrate that (1) machines can learn soft prompts better than hard prompts composed by human, (2) frozen LLMs have good few-shot learning ability and generalizability for cross-institution applications, (3) frozen LLMs reduce computing cost to 2.5 ∼ 6 % of previous methods using unfrozen LLMs, and (4) frozen LLMs require large models (e.g., over several billions of parameters) for good performance.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.