Evidence map›Paper›PMID 39557209›Full record

ArticleJournal of biomedical informatics2024

Importance of variables from different time frames for predicting self-harm using health system data.

Charles J Wolock, Brian D Williamson, Susan M Shortreed, Gregory E Simon, Karen J Coleman, Rodney Yeargans, Brian K Ahmedani, Yihe Daida, Frances L Lynch, Rebecca C Rossom and 4 more

Abstract read
In one paragraph

Article in Journal of biomedical informatics, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 3 papers.

0numbers the graph read from it
0cells of the map it votes in
3citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

3 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

14 authors.

Charles J WolockDepartment of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, 423 Guardian Dr., Philadelphia, PA, 19104, USA. Electronic address: cwolock@upenn.edu.
Brian D WilliamsonKaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Suite 1360, Seattle, WA, 98101, USA; Department of Biostatistics, University of Washington, 3980 15th Ave. NE, Box 351617, Seattle, WA, 98195, USA.
Susan M ShortreedKaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Suite 1360, Seattle, WA, 98101, USA; Department of Biostatistics, University of Washington, 3980 15th Ave. NE, Box 351617, Seattle, WA, 98195, USA.
Gregory E SimonKaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Suite 1360, Seattle, WA, 98101, USA; Department of Health Systems Science, Bernard J. Tyson Kaiser Permanente School of Medicine, 98 S. Los Robles Ave., Pasadena, CA, 91101, USA.
Karen J ColemanDepartment of Health Systems Science, Bernard J. Tyson Kaiser Permanente School of Medicine, 98 S. Los Robles Ave., Pasadena, CA, 91101, USA; Department of Research and Evaluation, Kaiser Permanente Southern California, 100 S. Los Robles Ave., Pasadena, CA, 91101, USA.
Rodney YeargansDepartment of Research and Evaluation, Kaiser Permanente Southern California, 100 S. Los Robles Ave., Pasadena, CA, 91101, USA.
Brian K AhmedaniCenter for Health Policy and Health Services Research, Henry Ford Health, One Ford Place - 3A, Detroit, MI, 48202, USA.
Yihe DaidaCenter for Integrated Health Care Research, Kaiser Permanente Hawaii, 501 Alakawa St., Suite 201, Honolulu, HI, 96817, USA.
Frances L LynchCenter for Health Research, Kaiser Permanente Northwest, 3800 N. Interstate Ave., Portland, OR, 97227, USA.
Rebecca C RossomHealthPartners Institute, 8170 33rd Ave. S., Bloomington, MN, 55425, USA.
Rebecca A ZiebellKaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Suite 1360, Seattle, WA, 98101, USA.
Maricela CruzKaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Suite 1360, Seattle, WA, 98101, USA; Department of Biostatistics, University of Washington, 3980 15th Ave. NE, Box 351617, Seattle, WA, 98195, USA.
Robert D WellmanKaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Suite 1360, Seattle, WA, 98101, USA.
R Yates ColeyKaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Suite 1360, Seattle, WA, 98101, USA; Department of Biostatistics, University of Washington, 3980 15th Ave. NE, Box 351617, Seattle, WA, 98195, USA.

Funding

SSRI's and SuicidalityU19MH092201 · NIMH · KAISER FOUNDATION RESEARCH INSTITUTE · PI SIMON, GREGORY E. · 2010 to 2018
$19.5M
Whose Depression Are We Measuring?: Considering the role of place on Black women’s depression outcomesU19MH121738 · NIMH · KAISER FOUNDATION RESEARCH INSTITUTE · PI SIMON, GREGORY E. · 2019 to 2023
$10.6M
Innovative methods to reduce racial and ethnic disparities in suicide risk predictionR01MH125821 · NIMH · KAISER FOUNDATION RESEARCH INSTITUTE · PI COLEY, REBECCA YATES · 2022 to 2025
$1.6M
NIMH NIH HHS R01 MH125821NIMH NIH HHS U19 MH092201NIMH NIH HHS U19 MH121738
6 · The paper itself

Abstract

objectiveSelf-harm risk prediction models developed using health system data (electronic health records and insurance claims information) often use patient information from up to several years prior to the index visit when the prediction is made. Measurements from some time periods may not be available for all patients. Using the framework of algorithm-agnostic variable importance, we study the predictive potential of variables corresponding to different time horizons prior to the index visit and demonstrate the application of variable importance techniques in the biomedical informatics setting. MATERIALS AND

methodsWe use variable importance to quantify the potential of recent (up to three months before the index visit) and distant (more than one year before the index visit) patient mental health information for predicting self-harm risk using data from seven health systems. We quantify importance as the decrease in predictiveness when the variable set of interest is excluded from the prediction task. We define predictiveness using discriminative metrics: area under the receiver operating characteristic curve (AUC), sensitivity, and positive predictive value.

resultsMental health predictors corresponding to the three months prior to the index visit show strong signal of importance; in one setting, excluding these variables decreased AUC from 0.85 to 0.77. Predictors corresponding to more distant information were less important. DISCUSSION: Predictors from the months immediately preceding the index visit are highly important. Implementation of self-harm prediction models may be challenging in settings where recent data are not completely available (e.g., due to lags in insurance claims processing) at the time a prediction is made.

conclusionClinically derived variables from different time frames exhibit varying levels of importance for predicting self-harm. Variable importance analyses can inform whether and how to implement risk prediction models into clinical practice given real-world data limitations. These analyses be applied more broadly in biomedical informatics research to provide insight into general clinical risk prediction tasks.

Indexed as

Electronic Health RecordsSelf-Injurious BehaviorAdultAlgorithmsArea Under CurveFemaleHumansMaleMedical InformaticsRisk AssessmentROC CurveTime FactorsClinical prediction modelsFeature importanceInsurance claims dataPredictive analyticsSuicide

Identifiers

PMID39557209
PMCPMC11891787

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.