Evidence map›Paper›PMID 24263289›Full record

Trial reportJournal of medical Internet research2013

Using multiple imputations to accommodate time-outs in online interventions.

Susan M Shortreed, Andy Bogart, Jennifer B McClure

Registry-linked trialAbstract readRandomized Controlled Trial
In one paragraph

Trial report in Journal of medical Internet research, 2013. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT00992264 (Optimizing an Online Motivational Tobacco Cessation Program), which is not on this map. Cited by 2 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
2citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT00992264 nacompletednot on this map

Optimizing an Online Motivational Tobacco Cessation Program

TypeinterventionalSponsorKaiser PermanenteRan2010 to 2013Enrolled1,865ConditionsSmokingArmsMessage Tone, Navigation, Proactive Outreach, Testimonials
3 · Its place in the literature

Who cites it

2 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Internet-based interventions for smoking cessation.The Cochrane database of systematic reviews · 2017
    Pooled it
  2. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Susan M ShortreedGroup Heatlh Research Institute, Biostatistics Unit, Seattle, WA, United States. shortreed.s@ghc.org.
Andy Bogart
Jennifer B McClure

Funding

Optimizing an online motivational tobacco cessation programR01CA138598 · NCI · KAISER FOUNDATION HEALTH PLAN OF WASHINGTON · PI MCCLURE, JENNIFER B · 2009 to 2012
$2.3M
NCI NIH HHS R01 CA138598
6 · The paper itself

Abstract

backgroundAccurately estimating the period of time that individuals are exposed to online intervention content is important for understanding program engagement. This can be calculated from time-stamped data reflecting navigation to and from individual webpages. Prolonged periods of inactivity are commonly handled with a time-out feature and assigned a prespecified exposure duration. Unfortunately, this practice can lead to biased results describing program exposure.

objectiveThe aim of the study was to describe how multiple imputations can be used to better account for the time spent viewing webpages that result in a prolonged period of inactivity or a time-out.

methodsTo illustrate this method, we present data on time-outs collected from the Q(2) randomized smoking cessation trial. For this analysis, we evaluate the effects on intervention exposure of receiving content written in a prescriptive versus motivational tone. Using multiple imputations, we created five complete datasets in which the time spent viewing webpages that resulted in a time-out were replaced with values estimated with imputation models. We calculated standard errors using Rubin's formulas to account for the variability due to the imputations. We also illustrate how current methods of accounting for time-outs (excluding timed-out page views or assigning an arbitrary viewing time) can influence conclusions about participant engagement.

resultsA total of 63.00% (1175/1865) of participants accessed the online intervention in the Q(2) trial. Of the 6592 unique page views, 683 (10.36%, 683/6592) resulted in a time-out. The median time spent viewing webpages that did not result in a time-out was 1.07 minutes. Assuming participants did not spend any time viewing a webpage that resulted in a time-out, no difference between the two message tones was observed (ratio of mean time online: 0.87, 95% CI 0.75-1.02). Assigning 30 minutes of viewing time to all page views that resulted in a time-out concludes that participants who received content in a motivational tone spent less time viewing content (ratio of mean time online: 0.86, 95% CI 0.77-0.98) than those participants who received content in a prescriptive tone. Using multiple imputations to account for time-outs concludes that there is no difference in participant engagement between the two message tones (ratio of mean time online: 0.87; 95% CI 0.75-1.01).

conclusionsThe analytic technique chosen can significantly affect conclusions about online intervention engagement. We propose a standardized methodology in which time spent viewing webpages that result in a time-out is treated as missing information and corrected with multiple imputations.

trial registrationClinicaltrials.gov NCT00992264; http://clinicaltrials.gov/ct2/show/NCT00992264 (Archived by WebCite at http://www.webcitation.org/6Kw5m8EkP).

Indexed as

InternetHumansMotivationSmoking CessationSocial Supportautomatic time-outbehavioral researchengagementInternetmultiple imputationsonline interventionssmoking cessationtime spent onlineutilization

Identifiers

PMID24263289
PMCPMC3841344

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.