ArticleJournal of the American Medical Informatics Association : JAMIA2023
Automated deidentification of radiology reports combining transformer and "hide in plain sight" rule-based methods.
Article in Journal of the American Medical Informatics Association : JAMIA, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 28 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
28 citing papers in PubMed.
- Mental Health Screening Before and After the 2020 COVID-19 Closures of US College Campuses: A Cross-Sectional Comparison Using Natural Language Processing.Public health reports (Washington, D.C. : 1974) · 2026Article
- Generalizable multilingual medical text anonymization using generative instruction tuning.Communications medicine · 2026Article
- Enhancing bone metastasis CT report analysis: a comparison of local and proprietary large language models for privacy and resource efficiency.BMC health services research · 2026Article
- Real-world performance evaluation of a commercial deep learning model for intracranial hemorrhage detection.NPJ digital medicine · 2025Article
- Article
- Leveraging large language models for structured information extraction from pathology reports.Journal of pathology informatics · 2025Article
- A Markov Chain Replacement Strategy for Surrogate Identifiers: Minimizing Re-Identification Risk While Preserving Text Reuse.Electronics · 2025Article
- RadGPT: A System Based on a Large Language Model That Generates Sets of Patient-Centered Materials to Explain Radiology Report Information.Journal of the American College of Radiology : JACR · 2025Article
- Best Practices for the Safe Use of Large Language Models and Other Generative AI in Radiology.Radiology · 2025Review
- Performance of Natural Language Processing versus International Classification of Diseases Codes in Building Registries for Patients With Fall Injury: Retrospective Analysis.JMIR medical informatics · 2025Article
- A textual dataset of de-identified health records in Spanish and Catalan for medical entity recognition and anonymization.Scientific data · 2025Article
- MC-MED, multimodal clinical monitoring in the emergency department.Scientific data · 2025Article
- Detecting emergencies in patient portal messages using large language models and knowledge graph-based retrieval-augmented generation.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Article
- Automated anonymization of radiology reports: comparison of publicly available natural language processing and large language models.European radiology · 2025Article
- Characterizing patient portal use of people with cognitive impairment and potentially inappropriate medications.Journal of the American Geriatrics Society · 2025Article
- Effective Structured Information Extraction from Chest Radiography Reports Using Open-Weights Large Language Models.Radiology · 2025Article
- Automated redaction of names in adverse event reports using transformer-based neural networks.BMC medical informatics and decision making · 2024Article
- Developing a Research Center for Artificial Intelligence in Medicine.Mayo Clinic proceedings. Digital health · 2024Article
- Year 2023 in Biomedical Natural Language Processing: a Tribute to Large Language Models and Generative AI.Yearbook of medical informatics · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
objectiveTo develop an automated deidentification pipeline for radiology reports that detect protected health information (PHI) entities and replaces them with realistic surrogates "hiding in plain sight." MATERIALS AND
methodsIn this retrospective study, 999 chest X-ray and CT reports collected between November 2019 and November 2020 were annotated for PHI at the token level and combined with 3001 X-rays and 2193 medical notes previously labeled, forming a large multi-institutional and cross-domain dataset of 6193 documents. Two radiology test sets, from a known and a new institution, as well as i2b2 2006 and 2014 test sets, served as an evaluation set to estimate model performance and to compare it with previously released deidentification tools. Several PHI detection models were developed based on different training datasets, fine-tuning approaches and data augmentation techniques, and a synthetic PHI generation algorithm. These models were compared using metrics such as precision, recall and F1 score, as well as paired samples Wilcoxon tests.
resultsOur best PHI detection model achieves 97.9 F1 score on radiology reports from a known institution, 99.6 from a new institution, 99.5 on i2b2 2006, and 98.9 on i2b2 2014. On reports from a known institution, it achieves 99.1 recall of detecting the core of each PHI span. DISCUSSION: Our model outperforms all deidentifiers it was compared to on all test sets as well as human labelers on i2b2 2014 data. It enables accurate and automatic deidentification of radiology reports.
conclusionsA transformer-based deidentification pipeline can achieve state-of-the-art performance for deidentifying radiology reports and other medical documents.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.