What scientific inferences can be made with randomized implementation rollout trials.

backgroundRandomized rollout trial designs, including stepped wedge designs, are commonly used to examine how well an evidence-based intervention or package is being implemented in community or healthcare settings. The multitude of implementation research questions and specific hypotheses suggest the need for diverse randomized rollout implementation trial designs, assignment principles and procedureds, and statistical modeling.

methodsWe separate key research questions and identify mixed effect models for randomized implementation rollout trials involving 1) a single implementation strategy that tests how this strategy varies over time and/or resources that are allocated, 2) comparison of two distinct implementation strategies, and 3) three distinct strategies or components tested in a single trial. Appropriate rollout designs, optimal assignment methods, and other design and analysis considerations are discussed for trials of up to three distinct implementation strategies.

resultsTo examine improvement in implementation outcomes we present a Fixed-Length Staggered Rollout Trial Design to examine how well a sustainment period continues to produce outcomes, The Rollout Implementation Optimization (ROIO) methodology illustrates testing for quality improvement. For comparing an existing to new strategy, we focus on a Stepped Wedge design, and for comparing two new strategies we describe a Head-to-Head Rollout trial design. To test for synergy between two components, we introduce a Head-to-Head Rollout trial design, and for testing an existing strategy to a new one followed by a sustainment period, we recommend using a Three-Phase Sequential Rollout Implementation trial design. Modeling choices are described, including options for specifying random effects that capture variations in site and clustering. We discuss comparisons of superiority versus non-inferiority testing and multiple contrasts. To support uses of these six designs and analyses, we provide computational code.

conclusionsThe large class of randomized rollout implementation trial designs provides rich opportunities to address research questions posed by implementation scientists. Balance in assigning sites to cohorts is important before random assignment to time of transition to a new implementation occurs. Specific hypotheses are tested with mixed effects models where fixed effects include comparisons of implementation conditions and random effects that account for variation in sites and clustering.