Evidence map›Paper›PMID 36959268›Full record

ArticleNPJ digital medicine2023

Complex modeling with detailed temporal predictors does not improve health records-based suicide risk prediction.

Susan M Shortreed, Rod L Walker, Eric Johnson, Robert Wellman, Maricela Cruz, Rebecca Ziebell, R Yates Coley, Zimri S Yaseen, Sai Dharmarajan, Robert B Penfold and 5 more

Full text read
In one paragraph

Article in NPJ digital medicine, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 25 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
25citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

25 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. Article
  5. Life events extraction from healthcare notes for veteran acute suicide risk prediction.Journal of the American Medical Informatics Association : JAMIA · 2026
    Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Behavior of prediction performance metrics with rare events.Journal of clinical epidemiology · 2026
    Article
  12. Article
  13. Review
  14. Article
  15. Review
  16. Article
  17. Article
  18. Article
  19. Opportunities and Risks of Large Language Models in Psychiatry.NPP - digital psychiatry and neuroscience · 2024
    Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors.

Susan M ShortreedKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA. susan.m.shortreed@kp.org.ORCID http://orcid.org/0000-0001-7918-601X
Rod L WalkerKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.
Eric JohnsonKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.
Robert WellmanKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.ORCID http://orcid.org/0000-0002-1435-6134
Maricela CruzKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.
Rebecca ZiebellKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.
R Yates ColeyKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.
Zimri S YaseenU.S. Food and Drug Administration, Silver Spring, MD, USA.ORCID http://orcid.org/0000-0003-4103-2511
Sai DharmarajanU.S. Food and Drug Administration, Silver Spring, MD, USA.
Robert B PenfoldKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.ORCID http://orcid.org/0000-0003-4795-7902
Brian K AhmedaniCenter for Health Policy & Health Services Research, Henry Ford Health System, 1 Ford Place, Detroit, MI, 48202, USA.
Rebecca C RossomHealthPartners Institute, Division of Research, 8170 33rd Ave S, Minneapolis, MN, 55425, USA.
Arne BeckKaiser Permanente Colorado Institute for Health Research, 2550 S. Parker Road, Suite 200, Aurora, CO, 80014, USA.
Jennifer M BoggsKaiser Permanente Colorado Institute for Health Research, 2550 S. Parker Road, Suite 200, Aurora, CO, 80014, USA.
Greg E SimonKaiser Permanente Washington Health Research Institute, 1730 Minor Avenue, Ste 1600, Seattle, WA, 98101, USA.

Funding

SSRI's and SuicidalityU19MH092201 · NIMH · KAISER FOUNDATION RESEARCH INSTITUTE · PI SIMON, GREGORY E. · 2010 to 2018
$19.5M
Whose Depression Are We Measuring?: Considering the role of place on Black women’s depression outcomesU19MH121738 · NIMH · KAISER FOUNDATION RESEARCH INSTITUTE · PI SIMON, GREGORY E. · 2019 to 2023
$10.6M
NIMH NIH HHS U19 MH092201NIMH NIH HHS U19 MH121738U.S. Department of Health & Human Services | U.S. Food and Drug Administration (U.S. Food & Drug Administration) HHSF223201810201C
6 · The paper itself

Abstract

Suicide risk prediction models can identify individuals for targeted intervention. Discussions of transparency, explainability, and transportability in machine learning presume complex prediction models with many variables outperform simpler models. We compared random forest, artificial neural network, and ensemble models with 1500 temporally defined predictors to logistic regression models. Data from 25,800,888 mental health visits made by 3,081,420 individuals in 7 health systems were used to train and evaluate suicidal behavior prediction models. Model performance was compared across several measures. All models performed well (area under the receiver operating curve [AUC]: 0.794-0.858). Ensemble models performed best, but improvements over a regression model with 100 predictors were minimal (AUC improvements: 0.006-0.020). Results are consistent across performance metrics and subgroups defined by race, ethnicity, and sex. Our results suggest simpler parametric models, which are easier to implement as part of routine clinical practice, perform comparably to more complex machine learning methods.

Identifiers

PMID36959268
PMCPMC10036475

What OpenQuestion holds

Textfull text, public
LicenceCC BY
measurements read101
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.