ArticleJMIR research protocols2020
Notifications to Improve Engagement With an Alcohol Reduction App: Protocol for a Micro-Randomized Trial.
Article in JMIR research protocols, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 12 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
12 citing papers in PubMed, 31 citations in OpenAlex.
- How Notifications Affect Engagement With a Behavior Change App: Results From a Micro-Randomized Trial.JMIR mHealth and uHealth · 2023Trial
- Reminder Strategies to Improve Meal-Logging Adherence: Protocol for a Microrandomized Trial.JMIR research protocols · 2026Article
- Designing digital health interventions with causal inference and multi-armed bandits: a review.Frontiers in digital health · 2025Review
- The Digital Therapeutics Real-World Evidence Framework: An Approach for Guiding Evidence-Based Digital Therapeutics Design, Development, Testing, and Monitoring.Journal of medical Internet research · 2024Article
- Sample size considerations for micro-randomized trials with binary proximal outcomes.Statistics in medicine · 2023Article
- Participant Engagement in Microrandomized Trials of mHealth Interventions: Scoping Review.JMIR mHealth and uHealth · 2023Article
- MCMTC: A Pragmatic Framework for Selecting an Experimental Design to Inform the Development of Digital Interventions.Frontiers in digital health · 2022Article
- Optimizing a Just-in-Time Adaptive Intervention to Improve Dietary Adherence in Behavioral Obesity Treatment: Protocol for a Microrandomized Trial.JMIR research protocols · 2021Article
- Power Constrained Bandits.Proceedings of machine learning research · 2021Article
- The impact of celebrity influence and national media coverage on users of an alcohol reduction app: a natural experiment.BMC public health · 2021Article
- Engagement With a Behavior Change App for Alcohol Reduction: Data Visualization for Longitudinal Observational Study.Journal of medical Internet research · 2020Observational
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors at 4 institutions in 2 countries.
Funding
Abstract
backgroundDrink Less is a behavior change app that aims to help users in the general adult population reduce hazardous and harmful alcohol consumption. The app includes a daily push notification, delivered at 11 am, asking users to "Please complete your mood and drinking diaries." Previous analysis of Drink Less engagement data suggests the current notification strongly influences how users engage with the app in the subsequent hour. To exploit a potential increase of vulnerability of excess drinking and opportunity to engage with the app in the evenings, we changed the delivery time from 11 am to 8 pm. We now aim to further optimise the content and sequence of notifications, testing 30 new evidence-informed notifications targeting the user's perceived usefulness of the app.
objectiveThe primary objective is to assess whether sending a notification at 8 pm increases behavioral engagement (opening the app) in the subsequent hour. Secondary objectives include comparing the effect of the new bank of messages with the standard message and effect moderation over time. We also aim to more generally understand the role notifications have on the overall duration, depth, and frequency of engagement with Drink Less over the first 30 days after download.
methodsThis is a protocol for a micro-randomized trial with two additional parallel arms. Inclusion criteria are Drink Less users who (1) consent to participate in the trial; (2) self-report a baseline Alcohol Use Disorders Identification Test score of 8 or above; (3) reside in the United Kingdom; (4) age ≥18 years and; (5) report interest in drinking less alcohol. In the micro-randomized trial, participants will be randomized daily at 8 pm to receive no notification, a notification with text from the new message bank, or the standard message. The primary outcome is the time-varying, binary outcome of "Did the user open the app in the hour from 8 pm to 9 pm?". The primary analysis will estimate the marginal relative risk for the notifications using an estimator developed for micro-randomized trials with binary outcomes. Participants randomized to the parallel arms will receive no notifications (Secondary Arm A), or the standard notification delivered daily at 11 am (Secondary Arm B) over 30 days, allowing the comparison of overall engagement between different notification delivery strategies.
resultsApproval was granted by the University College of London's Departmental Research Ethics Committee (CEHP/2016/556) on October 11, 2019, and The London School of Hygiene and Tropical Medicine Interventions Research Ethics Committee (17929) on November 27, 2019. Recruitment began on January 2, 2020, and is ongoing.
conclusionsUnderstanding how push notifications may impact engagement with a behavior change app can lead to further improvements in engagement, and ultimately help users reduce their alcohol consumption. This understanding may also be generalizable to other apps that target a variety of behavior changes. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/18690.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.