ArticlePharmacoepidemiology and drug safety2020
The use of natural language processing to identify vaccine-related anaphylaxis at five health care systems in the Vaccine Safety Datalink.
Article in Pharmacoepidemiology and drug safety, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed.
- Identifying anaphylaxis using weakly-supervised prediction models and natural language processing.medRxiv : the preprint server for health sciences · 2026Article
- Identification and validation of respiratory virus immunization using natural language processing.Frontiers in digital health · 2026Article
- Protocol for Designing a Model to Predict the Likelihood of Psychosis From Electronic Health Records Using Natural Language Processing and Machine Learning.The Permanente journal · 2024Article
- A general framework for developing computable clinical phenotype algorithms.Journal of the American Medical Informatics Association : JAMIA · 2024Article
- The Impact of Artificial Intelligence on Allergy Diagnosis and Treatment.Current allergy and asthma reports · 2024Review
- Research data warehouse: using electronic health records to conduct population-based observational studies.JAMIA open · 2023Article
- Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning.American journal of epidemiology · 2023Article
- Measurement of provider fidelity to immunization guidelines: a mixed-methods study on the feasibility of documenting patient refusals of the human papillomavirus vaccine.BMC medical informatics and decision making · 2022Observational
- Information systems for vaccine safety surveillance.Human vaccines & immunotherapeutics · 2022Review
- Identification of Preterm Labor Evaluation Visits and Extraction of Cervical Length Measures from Electronic Health Records Within a Large Integrated Health Care System: Algorithm Development and Validation.JMIR medical informatics · 2022Article
- Identifying Cases of Shoulder Injury Related to Vaccine Administration (SIRVA) in the United States: Development and Validation of a Natural Language Processing Method.JMIR public health and surveillance · 2022Article
- Risk for Shoulder Conditions After Vaccination: A Population-Based Study Using Real-World Data.Annals of internal medicine · 2022Article
- Identifying Suicidal Ideation and Attempt From Clinical Notes Within a Large Integrated Health Care System.The Permanente journal · 2022Article
- COVID-19 Vaccine in Pregnant and Lactating Women: A Review of Existing Evidence and Practice Guidelines.Infectious disease reports · 2021Review
- The coronavirus disease 2019 vaccine in pregnancy: risks, benefits, and recommendations.American journal of obstetrics and gynecology · 2021Review
- Natural Language Processing for Surveillance of Cervical and Anal Cancer and Precancer: Algorithm Development and Split-Validation Study.JMIR medical informatics · 2020Article
- Challenges and opportunities for public health made possible by advances in natural language processing.Canada communicable disease report = Releve des maladies transmissibles au Canada · 2020Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
17 authors.
Funding
Abstract
purposeThe objective was to develop a natural language processing (NLP) algorithm to identify vaccine-related anaphylaxis from plain-text clinical notes, and to implement the algorithm at five health care systems in the Vaccine Safety Datalink.
methodsThe NLP algorithm was developed using an internal NLP tool and training dataset of 311 potential anaphylaxis cases from Kaiser Permanente Southern California (KPSC). We applied the algorithm to the notes of another 731 potential cases (423 from KPSC; 308 from other sites) with relevant codes (ICD-9-CM diagnosis codes for anaphylaxis, vaccine adverse reactions, and allergic reactions; Healthcare Common Procedure Coding System codes for epinephrine administration). NLP results were compared against a reference standard of chart reviewed and adjudicated cases. The algorithm was then separately applied to the notes of 6 427 359 KPSC vaccination visits (9 402 194 vaccine doses) without relevant codes.
resultsAt KPSC, NLP identified 12 of 16 true vaccine-related cases and achieved a sensitivity of 75.0%, specificity of 98.5%, positive predictive value (PPV) of 66.7%, and negative predictive value of 99.0% when applied to notes of patients with relevant diagnosis codes. NLP did not identify the five true cases at other sites. When NLP was applied to the notes of KPSC patients without relevant codes, it captured eight additional true cases confirmed by chart review and adjudication.
conclusionsThe current study demonstrated the potential to apply rule-based NLP algorithms to clinical notes to identify anaphylaxis cases. Increasing the size of training data, including clinical notes from all participating study sites in the training data, and preprocessing the clinical notes to handle special characters could improve the performance of the NLP algorithms. We recommend adding an NLP process followed by manual chart review in future vaccine safety studies to improve sensitivity and efficiency.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.