ArticleJournal of the American Medical Informatics Association : JAMIA2013
Large-scale evaluation of automated clinical note de-identification and its impact on information extraction.
Article in Journal of the American Medical Informatics Association : JAMIA, 2013. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 41 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
41 citing papers in PubMed, 88 citations in OpenAlex.
- Extracting Social Determinants of Health from Dental Clinical Notes.Applied clinical informatics · 2025Article
- Evaluating the accuracy of automated and semi-automated anonymization tools for unstructured health records.Surgical neurology international · 2025Article
- Supporting the working life exposome: Annotating occupational exposure for enhanced literature search.PloS one · 2024Article
- Using Clinician-Patient WeChat Group Communication Data to Identify Symptom Burdens in Patients With Uterine Fibroids Under Focused Ultrasound Ablation Surgery Treatment: Qualitative Study.JMIR formative research · 2023Article
- De-identification of free text data containing personal health information: a scoping review of reviews.International journal of population data science · 2023Article
- Investigation of the Utility of Features in a Clinical De-identification Model: A Demonstration Using EHR Pathology Reports for Advanced NSCLC Patients.Frontiers in digital health · 2022Article
- Supporting COVID-19 Disparity Investigations with Dynamically Adjusting Case Reporting Policies.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2022Article
- Improving domain adaptation in de-identification of electronic health records through self-training.Journal of the American Medical Informatics Association : JAMIA · 2021Article
- Privacy Protection and Secondary Use of Health Data: Strategies and Methods.BioMed research international · 2021Review
- Privacy-Preserving Deep Learning for the Detection of Protected Health Information in Real-World Data: Comparative Evaluation.JMIR formative research · 2020Article
- Protected Health Information filter (Philter): accurately and securely de-identifying free-text clinical notes.NPJ digital medicine · 2020Article
- Detecting conversation topics in primary care office visits from transcripts of patient-provider interactions.Journal of the American Medical Informatics Association : JAMIA · 2019Article
- A concept-wide association study to identify potential risk factors for nonadherence among prevalent users of antihypertensives.Pharmacoepidemiology and drug safety · 2019Article
- Impact of De-Identification on Clinical Text Classification Using Traditional and Deep Learning Classifiers.Studies in health technology and informatics · 2019Article
- Efficient Active Learning for Electronic Medical Record De-identification.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2019Article
- Automated Risk Assessment for School Violence: a Pilot Study.The Psychiatric quarterly · 2018Article
- Use of an electronic health record data sharing system for identifying current contraceptive use within the WWAMI region Practice and Research Network.Contraception · 2018Article
- Phelan-McDermid syndrome data network: Integrating patient reported outcomes with clinical notes and curated genetic reports.American journal of medical genetics. Part B, Neuropsychiatric genetics : the official publication of the International Society of Psychiatric Genetics · 2018Article
- De-identification of clinical notes via recurrent neural network and conditional random field.Journal of biomedical informatics · 2017Article
- Leveraging Health Information Exchange to Construct a Registry for Traumatic Brain Injury, Spinal Cord Injury and Stroke in Indiana.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2017Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors at 3 institutions in 1 country.
Funding
Abstract
objective(1) To evaluate a state-of-the-art natural language processing (NLP)-based approach to automatically de-identify a large set of diverse clinical notes. (2) To measure the impact of de-identification on the performance of information extraction algorithms on the de-identified documents. MATERIAL AND
methodsA cross-sectional study that included 3503 stratified, randomly selected clinical notes (over 22 note types) from five million documents produced at one of the largest US pediatric hospitals. Sensitivity, precision, F value of two automated de-identification systems for removing all 18 HIPAA-defined protected health information elements were computed. Performance was assessed against a manually generated 'gold standard'. Statistical significance was tested. The automated de-identification performance was also compared with that of two humans on a 10% subsample of the gold standard. The effect of de-identification on the performance of subsequent medication extraction was measured.
resultsThe gold standard included 30 815 protected health information elements and more than one million tokens. The most accurate NLP method had 91.92% sensitivity (R) and 95.08% precision (P) overall. The performance of the system was indistinguishable from that of human annotators (annotators' performance was 92.15%(R)/93.95%(P) and 94.55%(R)/88.45%(P) overall while the best system obtained 92.91%(R)/95.73%(P) on same text). The impact of automated de-identification was minimal on the utility of the narrative notes for subsequent information extraction as measured by the sensitivity and precision of medication name extraction. DISCUSSION AND
conclusionNLP-based de-identification shows excellent performance that rivals the performance of human annotators. Furthermore, unlike manual de-identification, the automated approach scales up to millions of documents quickly and inexpensively.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.