Evidence map›Paper›PMID 35608886›Full record

ArticleJMIR public health and surveillance2022

Identifying Cases of Shoulder Injury Related to Vaccine Administration (SIRVA) in the United States: Development and Validation of a Natural Language Processing Method.

Chengyi Zheng, Jonathan Duffy, In-Lu Amy Liu, Lina S Sy, Ronald A Navarro, Sunhea S Kim, Denison S Ryan, Wansu Chen, Lei Qian, Cheryl Mercado and 1 more

Open access · goldAbstract readValidation Study
In one paragraph

Article in JMIR public health and surveillance, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers.

0numbers the graph read from it
0cells of the map it votes in
9citing papers in PubMed
3.6field-weighted citation impact, top 7% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

9 citing papers in PubMed, 14 citations in OpenAlex.

  1. Article
  2. Article
  3. Article
  4. Observational
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

11 authors at 2 institutions in 1 country.

Chengyi ZhengDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0002-4194-0029
Jonathan DuffyImmunization Safety Office, Centers for Disease Control and Prevention, Atlanta, GA, United States.ORCID 0000-0002-8060-0940
In-Lu Amy LiuDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0002-7760-7068
Lina S SyDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0003-2762-9266
Ronald A NavarroKaiser Permanente South Bay Medical Center, Harbor City, CA, United States.ORCID 0000-0003-1869-6440
Sunhea S KimDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0002-0228-9341
Denison S RyanDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0001-8440-3859
Wansu ChenDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0002-3486-7468
Lei QianDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0001-8001-3992
Cheryl MercadoDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0002-2981-5997
Steven J JacobsenDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States.ORCID 0000-0002-8174-8533
Kaiser Permanente · USCenters for Disease Control and Prevention · US

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundShoulder injury related to vaccine administration (SIRVA) accounts for more than half of all claims received by the National Vaccine Injury Compensation Program. However, due to the difficulty of finding SIRVA cases in large health care databases, population-based studies are scarce.

objectiveThe goal of the research was to develop a natural language processing (NLP) method to identify SIRVA cases from clinical notes.

methodsWe conducted the study among members of a large integrated health care organization who were vaccinated between April 1, 2016, and December 31, 2017, and had subsequent diagnosis codes indicative of shoulder injury. Based on a training data set with a chart review reference standard of 164 cases, we developed an NLP algorithm to extract shoulder disorder information, including prior vaccination, anatomic location, temporality and causality. The algorithm identified 3 groups of positive SIRVA cases (definite, probable, and possible) based on the strength of evidence. We compared NLP results to a chart review reference standard of 100 vaccinated cases. We then applied the final automated NLP algorithm to a broader cohort of vaccinated persons with a shoulder injury diagnosis code and performed manual chart confirmation on a random sample of NLP-identified definite cases and all NLP-identified probable and possible cases.

resultsIn the validation sample, the NLP algorithm had 100% accuracy for identifying 4 SIRVA cases and 96 cases without SIRVA. In the broader cohort of 53,585 vaccinations, the NLP algorithm identified 291 definite, 124 probable, and 52 possible SIRVA cases. The chart-confirmation rates for these groups were 95.5% (278/291), 67.7% (84/124), and 17.3% (9/52), respectively.

conclusionsThe algorithm performed with high sensitivity and reasonable specificity in identifying positive SIRVA cases. The NLP algorithm can potentially be used in future population-based studies to identify this rare adverse event, avoiding labor-intensive chart review validation.

Indexed as

Shoulder InjuriesVaccinationVaccinesAlgorithmsHumansNatural Language ProcessingUnited StatesVaccinesartificial intelligencebig datacausal relationEHRelectronic health recordshealthinformaticsnatural language processingNLPpharmacovigilancepopulation healthreal-world datashoulder injury related to vaccine administrationSIRVAtemporal relationvaccinesvaccine safety

Identifiers

PMID35608886
PMCPMC9175103
OpenAlexW4282914414

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.