Evidence map›Paper›PMID 39375687›Full record

ArticleBMC medical informatics and decision making2024

A hybrid framework with large language models for rare disease phenotyping.

Jinge Wu, Hang Dong, Zexi Li, Haowei Wang, Runci Li, Arijit Patra, Chengliang Dai, Waqar Ali, Phil Scordis, Honghan Wu

Abstract read
In one paragraph

Article in BMC medical informatics and decision making, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 13 papers.

0numbers the graph read from it
0cells of the map it votes in
13citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

13 citing papers in PubMed.

  1. Review
  2. Review
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Jinge WuInstitute of Health Informatics, University College London, London, UK. jinge.wu.20@ucl.ac.uk.
Hang DongDepartment of Computer Science, University of Exeter, Exeter, UK.
Zexi LiThe Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK.
Haowei WangDivision of Medicine, University College London, London, UK.
Runci LiEGA- Institute for Women's Health, University College London, London, UK.
Arijit PatraUCB Pharma UK, Slough, UK.
Chengliang DaiUCB Pharma UK, Slough, UK.
Waqar AliUCB Pharma UK, Slough, UK.
Phil ScordisUCB Pharma UK, Slough, UK.
Honghan WuInstitute of Health Informatics, University College London, London, UK. honghan.wu@ucl.ac.uk.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeRare diseases pose significant challenges in diagnosis and treatment due to their low prevalence and heterogeneous clinical presentations. Unstructured clinical notes contain valuable information for identifying rare diseases, but manual curation is time-consuming and prone to subjectivity. This study aims to develop a hybrid approach combining dictionary-based natural language processing (NLP) tools with large language models (LLMs) to improve rare disease identification from unstructured clinical reports.

methodsWe propose a novel hybrid framework that integrates the Orphanet Rare Disease Ontology (ORDO) and the Unified Medical Language System (UMLS) to create a comprehensive rare disease vocabulary. SemEHR, a dictionary-based NLP tool, is employed to extract rare disease mentions from clinical notes. To refine the results and improve accuracy, we leverage various LLMs, including LLaMA3, Phi3-mini, and domain-specific models like OpenBioLLM and BioMistral. Different prompting strategies, such as zero-shot, few-shot, and knowledge-augmented generation, are explored to optimize the LLMs' performance.

resultsThe proposed hybrid approach demonstrates superior performance compared to traditional NLP systems and standalone LLMs. LLaMA3 and Phi3-mini achieve the highest F1 scores in rare disease identification. Few-shot prompting with 1-3 examples yields the best results, while knowledge-augmented generation shows limited improvement. Notably, the approach uncovers a significant number of potential rare disease cases not documented in structured diagnostic records, highlighting its ability to identify previously unrecognized patients.

conclusionThe hybrid approach combining dictionary-based NLP tools with LLMs shows great promise for improving rare disease identification from unstructured clinical reports. By leveraging the strengths of both techniques, the method demonstrates superior performance and the potential to uncover hidden rare disease cases. Further research is needed to address limitations related to ontology mapping and overlapping case identification, and to integrate the approach into clinical practice for early diagnosis and improved patient outcomes.

Indexed as

Natural Language ProcessingRare DiseasesUnified Medical Language SystemBiological OntologiesElectronic Health RecordsHumansPhenotypeElectronic health recordLarge language modelNatural language processingPhenotyping

Identifiers

PMID39375687
PMCPMC11460004

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.