ArticleJournal of biomedical informatics2021
A strategy for validation of variables derived from large-scale electronic health record data.
Article in Journal of biomedical informatics, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed, 1 synthesis or guideline pooled it, 27 citations in OpenAlex.
- A Systematic Review to Summarize and Critically Appraise Existing Phenotype Libraries Using Electronic Health Records.Pharmacoepidemiology and drug safety · 2026Pooled it
- Using electronic health record data to identify incident uterine fibroids and endometriosis within a large, urban academic medical center: a validation study.American journal of epidemiology · 2026Article
- Leveraging Natural Language Processing to Accurately Identify Histologically Confirmed Esophageal and Gastric Premalignant and Malignant Conditions Using a Large-Scale Health-Record Linked Genomic Biobank.Gastro hep advances · 2026Article
- Large language models for extracting histopathologic diagnoses of colorectal cancer and dysplasia from electronic health records.BMJ open gastroenterology · 2025Article
- Delayed Venous Thromboembolism Diagnosis and Mortality Risk.JAMA network open · 2025Article
- Using electronic medical records to analyze outpatient visits of persons with epilepsy during the pandemic-experience from a low middle income country.Acta epileptologica · 2025Article
- Protocol for a type 3 hybrid implementation cluster randomized clinical trial to evaluate the effect of patient and clinician nudges to advance the use of genomic medicine across a diverse health system.Implementation science : IS · 2024Article
- Ascertainment ofGastro hep advances · 2024Article
- Development and Validation of the Veterans Affairs Eosinophilic Esophagitis Cohort.Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association · 2023Article
- Development and Validation of Quality Measures for Testosterone Prescribing.Journal of the Endocrine Society · 2023Article
- Optimal Acute Kidney Injury Algorithm for Detecting Acute Kidney Injury at Emergency Department Presentation.Kidney medicine · 2023Article
- Reducing disparities and achieving health equity in colorectal cancer screening.Techniques and innovations in gastrointestinal endoscopy · 2023Article
- Clinical Research Informatics.Yearbook of medical informatics · 2022Review
- Monocyte distribution width as part of a broad pragmatic sepsis screen in the emergency department.Journal of the American College of Emergency Physicians open · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors at 2 institutions in 1 country.
Funding
Abstract
purposeStandardized approaches for rigorous validation of phenotyping from large-scale electronic health record (EHR) data have not been widely reported. We proposed a methodologically rigorous and efficient approach to guide such validation, including strategies for sampling cases and controls, determining sample sizes, estimating algorithm performance, and terminating the validation process, hereafter referred to as the San Diego Approach to Variable Validation (SDAVV).
methodsWe propose sample size formulae which should be used prior to chart review, based on pre-specified critical lower bounds for positive predictive value (PPV) and negative predictive value (NPV). We also propose a stepwise strategy for iterative algorithm development/validation cycles, updating sample sizes for data abstraction until both PPV and NPV achieve target performance.
resultsWe applied the SDAVV to a Department of Veterans Affairs study in which we created two phenotyping algorithms, one for distinguishing normal colonoscopy cases from abnormal colonoscopy controls and one for identifying aspirin exposure. Estimated PPV and NPV both reached 0.970 with a 95% confidence lower bound of 0.915, estimated sensitivity was 0.963 and specificity was 0.975 for identifying normal colonoscopy cases. The phenotyping algorithm for identifying aspirin exposure reached a PPV of 0.990 (a 95% lower bound of 0.950), an NPV of 0.980 (a 95% lower bound of 0.930), and sensitivity and specificity were 0.960 and 1.000.
conclusionsA structured approach for prospectively developing and validating phenotyping algorithms from large-scale EHR data can be successfully implemented, and should be considered to improve the quality of "big data" research.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.