Evidence map›Paper›PMID 42082685›Full record

ArticleNPJ digital medicine2026

Effective monitoring of online AI decision-making algorithms in just-in-time adaptive interventions.

Anna L Trella, Susobhan Ghosh, Erin E Bonar, Lara Coughlin, Finale Doshi-Velez, Yongyi Guo, Pei-Yao Hung, Inbal Nahum-Shani, Vivek Shetty, Maureen Walton and 3 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in NPJ digital medicine, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

13 authors.

Anna L Trella *Department of Computer Science, Harvard University, Cambridge, MA, USA.
Susobhan Ghosh *Department of Computer Science, Harvard University, Cambridge, MA, USA. susobhan_ghosh@g.harvard.edu.
Erin E BonarDepartment of Psychiatry, University of Michigan, Ann Arbor, MI, USA.
Lara CoughlinDepartment of Psychiatry, University of Michigan, Ann Arbor, MI, USA.
Finale Doshi-VelezDepartment of Computer Science, Harvard University, Cambridge, MA, USA.
Yongyi GuoDepartment of Statistics, University of Wisconsin-Madison, Madison, WI, USA.
Pei-Yao HungInstitute for Social Research, University of Michigan, Ann Arbor, MI, USA.
Inbal Nahum-ShaniInstitute for Social Research, University of Michigan, Ann Arbor, MI, USA.
Vivek ShettySchool of Dentistry & Engineering, University of California, Los Angeles, Los Angeles, CA, USA.
Maureen WaltonDepartment of Psychiatry, University of Michigan, Ann Arbor, MI, USA.
Iris YanDepartment of Computer Science, Harvard University, Cambridge, MA, USA.
Kelly W ZhangMathematics Department, Imperial College London, South Kensington, London, UK.
Susan A MurphyDepartment of Computer Science, Harvard University, Cambridge, MA, USA.

Funding

Pilot and Mentoring CoreP50DA054039 · NIDA · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI LINDA M COLLINS, SUSAN A MURPHY · 2021 to 2026
$18.2M
TR&D3 - Rapid Translation of AI-powered Temporally Precise mHealth Interventions via Efficient and Embeddable Trustworthy Biomarker ImplementationsP41EB028242 · NIBIB · UNIVERSITY OF MEMPHIS · PI Santosh Kumar · 2020 to 2026
$9.5M
Personalized Digital Behavior Change Interventions to Promote Oral HealthUH3DE028723 · NIDCR · UNIVERSITY OF CALIFORNIA LOS ANGELES · PI VIVEK SHETTY · 2022 to 2026
$3.8M
Data-driven subtyping in major depressive disorderR01MH123804 · NIMH · MASSACHUSETTS GENERAL HOSPITAL · PI PERLIS, ROY H. · 2021 to 2024
$3.1M
Novel use of mHealth data to identify states of vulnerability and receptivity to JITAIs SupplementU01CA229437 · NCI · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI NAHUM-SHANI, INBAL BILLIE, WETTER, DAVID W · 2018 to 2022
$2.8M
Personalized Digital Behavior Change Interventions to Promote Oral HealthUG3DE028723 · NIDCR · UNIVERSITY OF CALIFORNIA LOS ANGELES · PI SHETTY, VIVEK · 2019 to 2019
$460k
NCI NIH HHS U01 CA229437NIBIB NIH HHS P41 EB028242NIDA NIH HHS P50 DA054039NIDCR NIH HHS UG3 DE028723NIDCR NIH HHS UH3 DE028723NIH HHS IUG3DE028723, P50DA054039, P41EB028242, U01CA229437, UH3DE028723, and R01MH123804NIH HHS P50DA054039 and P41EB028242NIMH NIH HHS R01 MH123804
6 · The paper itself

Abstract

Monitoring just-in-time adaptive interventions (JITAIs) is important both during trialing and when the intervention is deployed in a broader healthcare program. While there is increasing interest in using artificial intelligence (AI) algorithms in JITAIs, these algorithms introduce additional complexity that requires additional monitoring. In this paper, we provide guidelines for monitoring online AI decision-making algorithms. Our guidelines include: (1) identifying potential issues, categorizing them by severity (red, yellow, and green), and (2) developing fallback methods (pre-specified procedures that are executed when an issue occurs). To make ideas concrete, we discuss algorithm monitoring systems in two case studies. In both, the monitoring systems detected real-time issues, and fallback methods both safeguarded participants and ensured quality data for post-deployment data analysis to further refine the JITAI. These guidelines and findings give teams the confidence to include online AI decision-making algorithms in JITAIs.

Identifiers

PMID42082685
PMCPMC13518862

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.