ArticlePloS one2024
Systematic evaluation of common natural language processing techniques to codify clinical notes.
Article in PloS one, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 22 citations in OpenAlex.
- The daily dose: Early usability of an LLM tool for patient summaries and trial matching in radiation oncology.Clinical and translational radiation oncology · 2026Article
- Adapting transformer-based encoder models for automated billing code assignment of current procedural terminology from pathology reports in multi-institutional settings.Journal of pathology informatics · 2026Article
- The Effect of Age, Sex, and Body Mass Index on Spinopelvic Balance Throughout Childhood and Adolescence: An Automated 3D-Computed Tomography Study of 9,721 Pediatric Patients.Journal of the Pediatric Orthopaedic Society of North America · 2026Article
- Large language models for zero-shot procedure extraction in orthopedic surgery: a comparative evaluation.Scientific reports · 2026Article
- Automated abstraction of clinical parameters of multiple myeloma from real-world clinical notes using large language models.BMC medical informatics and decision making · 2026Article
- Artificial intelligence-assisted medical coding and DRG management: current applications, challenges, and future perspectives.Frontiers in medicine · 2026Review
- Risk Management of Large Language Model-Based Exercise and Health Guidance: A China-Anchored, Comparatively Informed Six-Dimensional Trigger Matrix and Lifecycle Governance Framework for the Wellness-to-SaMD Continuum.Risk management and healthcare policy · 2026Review
- Artificial intelligence agents in orthopaedics: Concepts, capabilities and the road ahead.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2025Review
- A practical guide to the implementation of artificial intelligence in orthopaedic research-Part 3: How orthopaedic research benefits from the implementation of artificial intelligence.Journal of experimental orthopaedics · 2025Review
- A practical guide to the implementation of AI in orthopaedic research-Part 5: Data management.Journal of experimental orthopaedics · 2025Review
- A scoping review of natural language processing in addressing medically inaccurate information: Errors, misinformation, and hallucination.Journal of biomedical informatics · 2025Article
- Mitigating the risk of health inequity exacerbated by large language models.NPJ digital medicine · 2025Article
- Article
- Prevalence and Predictors of Concomitant Meniscal Surgery During Pediatric and Adolescent ACL Reconstruction: Analysis of 4729 Patients Over 20 Years at a Tertiary-Care Regional Children's Hospital.Orthopaedic journal of sports medicine · 2024Article
- Predicting emergency department visits and hospitalizations for patients with heart failure in home healthcare using a time series risk model.Journal of the American Medical Informatics Association : JAMIA · 2023Article
- Disparities in cannabis use and documentation in electronic health records among children and young adults.NPJ digital medicine · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors at 1 institution in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Proper codification of medical diagnoses and procedures is essential for optimized health care management, quality improvement, research, and reimbursement tasks within large healthcare systems. Assignment of diagnostic or procedure codes is a tedious manual process, often prone to human error. Natural Language Processing (NLP) has been suggested to facilitate this manual codification process. Yet, little is known on best practices to utilize NLP for such applications. With Large Language Models (LLMs) becoming more ubiquitous in daily life, it is critical to remember, not every task requires that level of resource and effort. Here we comprehensively assessed the performance of common NLP techniques to predict current procedural terminology (CPT) from operative notes. CPT codes are commonly used to track surgical procedures and interventions and are the primary means for reimbursement. Our analysis of 100 most common musculoskeletal CPT codes suggest that traditional approaches can outperform more resource intensive approaches like BERT significantly (P-value = 4.4e-17) with average AUROC of 0.96 and accuracy of 0.97, in addition to providing interpretability which can be very helpful and even crucial in the clinical domain. We also proposed a complexity measure to quantify the complexity of a classification task and how this measure could influence the effect of dataset size on model's performance. Finally, we provide preliminary evidence that NLP can help minimize the codification error, including mislabeling due to human error.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.