ArticleJournal of medical Internet research2022
Development and Evaluation of a Natural Language Processing Annotation Tool to Facilitate Phenotyping of Cognitive Status in Electronic Health Records: Diagnostic Study.
Article in Journal of medical Internet research, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
18 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Natural language processing for geriatric syndromes: a systematic review of methods, applications, and challenges.BMC medical informatics and decision making · 2026Pooled it
- Natural Language Processing Applied to Psychiatric Clinical Notes: Scoping Review.JMIR medical informatics · 2026Article
- Accelerating real-world prediction and research in Alzheimer's: The M3AD study.Alzheimer's & dementia : the journal of the Alzheimer's Association · 2026Article
- Strategies for Safer Cefepime Use to Prevent Neurotoxicity Using the Electronic Health Record.Critical care explorations · 2026Article
- An Electronic Health Record Algorithm's Performance to Identify Cognitive Impairment in Primary Care.Journal of the American Medical Directors Association · 2026Article
- CARE-AD: a multi-agent large language model framework for Alzheimer's disease prediction using longitudinal clinical notes.NPJ digital medicine · 2025Article
- A GPT-4o-powered framework for identifying cognitive impairment stages in electronic health records.NPJ digital medicine · 2025Article
- Investigating Primary Care Indications to Improve the Quality of Electronic Health Record Data in Target Trial Emulation for Dementia.medRxiv : the preprint server for health sciences · 2025Article
- Natural language processing of electronic health records for early detection of cognitive decline: a systematic review.NPJ digital medicine · 2025Article
- Article
- Hybrid natural language processing tool for semantic annotation of medical texts in Spanish.BMC bioinformatics · 2025Article
- Extracting Cognitive Impairment Assessment Information From Unstructured Notes in Electronic Health Records Using Natural Language Processing Tools: Validation with Clinical Assessment Data.Clinical epidemiology · 2025Article
- Medical Marijuana Documentation Practices in Patient Electronic Health Records: Retrospective Observational Study Using Smart Data Elements and a Review of Medical Records.JMIR formative research · 2024Observational
- Using Natural Language Processing to Identify Home Health Care Patients at Risk for Diagnosis of Alzheimer's Disease and Related Dementias.Journal of applied gerontology : the official journal of the Southern Gerontological Society · 2024Article
- Extracting Critical Information from Unstructured Clinicians' Notes Data to Identify Dementia Severity Using a Rule-Based Approach: Feasibility Study.JMIR aging · 2024Article
- Deep Learning and Geriatric Mental Health.The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry · 2024Review
- Performance of an embedded algorithm to identify people with dementia in a clinical trial.Journal of the American Geriatrics Society · 2023Article
- Extraction of clinical phenotypes for Alzheimer's disease dementia from clinical notes using natural language processing.JAMIA open · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
22 authors.
Funding
Abstract
backgroundElectronic health records (EHRs) with large sample sizes and rich information offer great potential for dementia research, but current methods of phenotyping cognitive status are not scalable.
objectiveThe aim of this study was to evaluate whether natural language processing (NLP)-powered semiautomated annotation can improve the speed and interrater reliability of chart reviews for phenotyping cognitive status.
methodsIn this diagnostic study, we developed and evaluated a semiautomated NLP-powered annotation tool (NAT) to facilitate phenotyping of cognitive status. Clinical experts adjudicated the cognitive status of 627 patients at Mass General Brigham (MGB) health care, using NAT or traditional chart reviews. Patient charts contained EHR data from two data sets: (1) records from January 1, 2017, to December 31, 2018, for 100 Medicare beneficiaries from the MGB Accountable Care Organization and (2) records from 2 years prior to COVID-19 diagnosis to the date of COVID-19 diagnosis for 527 MGB patients. All EHR data from the relevant period were extracted; diagnosis codes, medications, and laboratory test values were processed and summarized; clinical notes were processed through an NLP pipeline; and a web tool was developed to present an integrated view of all data. Cognitive status was rated as cognitively normal, cognitively impaired, or undetermined. Assessment time and interrater agreement of NAT compared to manual chart reviews for cognitive status phenotyping was evaluated.
resultsNAT adjudication provided higher interrater agreement (Cohen κ=0.89 vs κ=0.80) and significant speed up (time difference mean 1.4, SD 1.3 minutes; P<.001; ratio median 2.2, min-max 0.4-20) over manual chart reviews. There was moderate agreement with manual chart reviews (Cohen κ=0.67). In the cases that exhibited disagreement with manual chart reviews, NAT adjudication was able to produce assessments that had broader clinical consensus due to its integrated view of highlighted relevant information and semiautomated NLP features.
conclusionsNAT adjudication improves the speed and interrater reliability for phenotyping cognitive status compared to manual chart reviews. This study underscores the potential of an NLP-based clinically adjudicated method to build large-scale dementia research cohorts from EHRs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.