Evidence map›Paper›PMID 37637073›Full record

ArticleProceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference2023

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care.

Anna L Trella, Kelly W Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A Murphy

Abstract read
In one paragraph

Article in Proceedings of the ... Innovative Applications of Artificial Intelligence Conference. Innovative Applications of Artificial Intelligence Conference, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers.

0numbers the graph read from it
0cells of the map it votes in
16citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

16 citing papers in PubMed.

  1. Trial
  2. Article
  3. Article
  4. Article
  5. Review
  6. Collaborative AI for precision neurorehabilitation: a roadmap.Journal of neuroengineering and rehabilitation · 2025
    Review
  7. Replicable Bandits for Digital Health Interventions.Statistical science : a review journal of the Institute of Mathematical Statistics · 2025
    Article
  8. Reinforcement Learning on Dyads to Enhance Medication Adherence.Artificial intelligence in medicine. Conference on Artificial Intelligence in Medicine (2005- ) · 2025
    Article
  9. Article
  10. Review
  11. A Deployed Online Reinforcement Learning Algorithm In An Oral Health Clinical Trial.Proceedings of the ... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence · 2025
    Article
  12. Article
  13. Review
  14. Article
  15. Online learning in bandits with predicted context.Proceedings of machine learning research · 2024
    Article
  16. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Anna L TrellaDepartment of Computer Science, Harvard University.
Kelly W ZhangDepartment of Computer Science, Harvard University.
Inbal Nahum-ShaniInstitute for Social Research, University of Michigan.
Vivek ShettySchools of Dentistry & Engineering, University of California, Los Angeles.
Finale Doshi-VelezDepartment of Computer Science, Harvard University.
Susan A MurphyDepartment of Computer Science, Harvard University.

Funding

Pilot and Mentoring CoreP50DA054039 · NIDA · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI LINDA M COLLINS, SUSAN A MURPHY · 2021 to 2026
$18.2M
TR&D3 - Rapid Translation of AI-powered Temporally Precise mHealth Interventions via Efficient and Embeddable Trustworthy Biomarker ImplementationsP41EB028242 · NIBIB · UNIVERSITY OF MEMPHIS · PI Santosh Kumar · 2020 to 2026
$9.5M
Novel Methods for Intensive Longitudinal Data in SMART Studies of Drug Abuse and HIVR01DA039901 · NIDA · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI ALMIRALL, DANIEL, NAHUM-SHANI, INBAL BILLIE · 2015 to 2024
$5.4M
Personalized Digital Behavior Change Interventions to Promote Oral HealthUH3DE028723 · NIDCR · UNIVERSITY OF CALIFORNIA LOS ANGELES · PI VIVEK SHETTY · 2022 to 2026
$3.8M
Data-driven subtyping in major depressive disorderR01MH123804 · NIMH · MASSACHUSETTS GENERAL HOSPITAL · PI PERLIS, ROY H. · 2021 to 2024
$3.1M
Novel use of mHealth data to identify states of vulnerability and receptivity to JITAIs SupplementU01CA229437 · NCI · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI NAHUM-SHANI, INBAL BILLIE, WETTER, DAVID W · 2018 to 2022
$2.8M
Personalized Digital Behavior Change Interventions to Promote Oral HealthUG3DE028723 · NIDCR · UNIVERSITY OF CALIFORNIA LOS ANGELES · PI SHETTY, VIVEK · 2019 to 2019
$460k
NCI NIH HHS U01 CA229437NIBIB NIH HHS P41 EB028242NIDA NIH HHS P50 DA054039NIDA NIH HHS R01 DA039901NIDCR NIH HHS UG3 DE028723NIDCR NIH HHS UH3 DE028723NIMH NIH HHS R01 MH123804
6 · The paper itself

Abstract

While dental disease is largely preventable, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of current actions on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been designed to run stably and autonomously in a constrained, real-world setting characterized by highly noisy, sparse data. We address this challenge by designing a quality reward that maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics. To the best of our knowledge, Oralytics is the first mobile health study utilizing an RL algorithm designed to prevent dental disease by optimizing the delivery of motivational messages supporting oral self-care behaviors.

Identifiers

PMID37637073
PMCPMC10457015

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.