Evidence map›Paper›PMID 33906470›Full record

ArticleClinical and applied thrombosis/hemostasis : official journal of the International Academy of Clinical and Applied Thrombosis/Hemostasis

Natural Language Processing Performance for the Identification of Venous Thromboembolism in an Integrated Healthcare System.

Bela Woller, Austin Daw, Valerie Aston, Jim Lloyd, Greg Snow, Scott M Stevens, Scott C Woller, Peter Jones, Joseph Bledsoe

Open access · goldAbstract read
In one paragraph

Article in Clinical and applied thrombosis/hemostasis : official journal of the International Academy of Clinical and Applied Thrombosis/Hemostasis. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers.

0numbers the graph read from it
0cells of the map it votes in
16citing papers in PubMed
3.9field-weighted citation impact, top 5% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

16 citing papers in PubMed, 28 citations in OpenAlex.

  1. Article
  2. Article
  3. Article
  4. Artificial intelligence in clinical thrombosis and hemostasis: A review.Research and practice in thrombosis and haemostasis · 2025
    Review
  5. Review
  6. Article
  7. Article
  8. Article
  9. Review
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. D-dimer Thresholds to Exclude Pulmonary Embolism among COVID-19 Patients in the Emergency Department: Derivation with Independent Validation.Clinical and applied thrombosis/hemostasis : official journal of the International Academy of Clinical and Applied Thrombosis/Hemostasis
    Article
  16. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors at 4 institutions in 1 country.

Bela Woller2456Loyola University Chicago, Undergraduate Education, Chicago, IL, USA.
Austin DawUniversity of Colorado Health Sciences Center, Office of Human Research, Aurora, CO, USA.
Valerie Aston98078Intermountain Healthcare, Office of Research, Acute Care Research, Salt Lake City, UT, USA.
Jim Lloyd98078Intermountain Healthcare, Informatics and Analytics, Salt Lake City, UT, USA.
Greg Snow98078Intermountain Healthcare, Office of Research, Statistical Data Center, Salt Lake City, UT, USA.
Scott M StevensDepartment of Medicine, 98078Intermountain Medical Center and University of Utah, Salt Lake City, UT, USA.
Scott C WollerDepartment of Medicine, 98078Intermountain Medical Center and University of Utah, Salt Lake City, UT, USA.ORCID https://orcid.org/0000-0002-2522-2705
Peter Jones98078Intermountain Healthcare, Enterprise Analytics, Salt Lake City, UT, USA.
Joseph BledsoeDepartment of Emergency Medicine, 98078Intermountain Healthcare, Salt Lake City, UT, USA.ORCID https://orcid.org/0000-0002-3005-2878
Intermountain Healthcare · USUniversity of Utah · USLoyola University Chicago · USUniversity of Colorado Health · US

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Real-time identification of venous thromboembolism (VTE), defined as deep vein thrombosis (DVT) and pulmonary embolism (PE), can inform a healthcare organization's understanding of these events and be used to improve care. In a former publication, we reported the performance of an electronic medical record (EMR) interrogation tool that employs natural language processing (NLP) of imaging studies for the diagnosis of venous thromboembolism. Because we transitioned from the legacy electronic medical record to the Cerner product, iCentra, we now report the operating characteristics of the NLP EMR interrogation tool in the new EMR environment. Two hundred randomly selected patient encounters for which the imaging report assessed by NLP that revealed VTE was present were reviewed. These included one hundred imaging studies for which PE was identified. These included computed tomography pulmonary angiography-CTPA, ventilation perfusion-V/Q scan, and CT angiography of the chest/ abdomen/pelvis. One hundred randomly selected comprehensive ultrasound (CUS) that identified DVT were also obtained. For comparison, one hundred patient encounters in which PE was suspected and imaging was negative for PE (CTPA or V/Q) and 100 cases of suspected DVT with negative CUS as reported by NLP were also selected. Manual chart review of the 400 charts was performed and we report the sensitivity, specificity, positive and negative predictive values of NLP compared with manual chart review. NLP and manual review agreed on the presence of PE in 99 of 100 cases, the presence of DVT in 96 of 100 cases, the absence of PE in 99 of 100 cases and the absence of DVT in all 100 cases. When compared with manual chart review, NLP interrogation of CUS, CTPA, CT angiography of the chest, and V/Q scan yielded a sensitivity = 93.3%, specificity = 99.6%, positive predictive value = 97.1%, and negative predictive value = 99%.

Indexed as

Natural Language ProcessingDelivery of Health Care, IntegratedFemaleHumansMaleMiddle AgedVenous Thromboembolismcomputerized tomography pulmonary angiographynatural language processingNLPPEpulmonary embolismvenous thromboembolic diseaseVTE

Identifiers

PMID33906470
PMCPMC8107936
OpenAlexW3157286760

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.