Addressing Generalizability in Clinical Named Entity Recognition: Federated Learning or Large Language Models?: A Case Study on Visual Acuity Extraction from US and UK Eye Institutes.

Quang N Nguyen, Honghan Wu, Nikolas Pontikos, Sophia Y Wang

Abstract readComparative Study

In one paragraph

Article in AMIA ... Annual Symposium proceedings. AMIA Symposium, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it

0cells of the map it votes in

0citing papers in PubMed

–field-weighted citation impact

1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors.

Quang N NguyenByers Eye Institute, Stanford University, Palo Alto, CA.

Honghan WuUCL Institute of Health Informatics, London, UK.

Nikolas PontikosUCL Institute of Ophthalmology, London, UK.

Sophia Y WangByers Eye Institute, Stanford University, Palo Alto, CA.

Funding

Stanford Vision Research CoreP30EY026877 · NEI · STANFORD UNIVERSITY · PI Jeffrey L Goldberg · 2017 to 2026

$8.0M

Improving Generalizability in Artificial Intelligence Algorithms for GlaucomaR01EY036893 · NEI · STANFORD UNIVERSITY · PI Sophia Ying Wang · 2025 to 2026

$1.3M

NEI NIH HHS P30 EY026877NEI NIH HHS R01 EY036893

6 · The paper itself

Abstract

Clinical Named Entity Recognition (NER) is vital for extracting structured data from clinical text, but ensuring model generalizability across institutions remains challenging. This study compares two approaches: (1) Federated Learning (FL), a privacy-preserving decentralized method, and (2) Large Language Models (LLMs) trained on diverse corpora. We evaluate Visual Acuity (VA) extraction from ophthalmology notes at Stanford (USA) and Moorfields Eye Hospital (UK), using BERT-based models, FL strategies (FedAvg, STWT), and LLMs (Llama-3-70B, Mixtral-8x7B). Results show that FL significantly improves generalization, with STWT outperforming FedAvg in stability and accuracy. LLMs demonstrate strong performance on MEH data but struggle with structured Stanford notes. These findings highlight FL's effectiveness for cross-institutional learning while revealing domain-specific limitations of LLMs, underscoring the need for tailored approaches to clinical NER.

Indexed as

Electronic Health RecordsFederated LearningInformation Storage and RetrievalLarge Language ModelsNatural Language ProcessingVisual AcuityHumansUnited KingdomUnited States

Identifiers

PMID41726531

PMCPMC12919464

What OpenQuestion holds

Textmetadata

Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.