Evidence map›Paper›PMID 39797386›Full record

ArticleClinical trials (London, England)2025

A multilevel framework for recruitment and retention in implementation trials: An illustrative example.

Nathaniel J Williams, Alexandra E Gomes, Nallely R Vega, Susan Esp, Mimi Choy-Brown, Rinad S Beidas

Abstract read
In one paragraph

Article in Clinical trials (London, England), 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.

0numbers the graph read from it
0cells of the map it votes in
1citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

1 citing paper in PubMed.

  1. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Nathaniel J WilliamsInstitute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA.ORCID 0000-0002-3948-7480
Alexandra E GomesInstitute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA.
Nallely R VegaInstitute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA.
Susan EspInstitute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA.
Mimi Choy-BrownSchool of Social Work, University of Minnesota, Saint Paul, MN, USA.
Rinad S BeidasDepartment of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.ORCID 0000-0001-6056-6071

Funding

Randomized trial of a leadership and organizational change strategy to improve the implementation and sustainment of digital measurement-based care in youth mental health servicesR01MH119127 · NIMH · BOISE STATE UNIVERSITY · PI WILLIAMS, NATHANIEL J. · 2019 to 2022
$2.8M
NIMH NIH HHS R01 MH119127
6 · The paper itself

Abstract

backgroundImplementation and hybrid effectiveness-implementation trials aspire to speed the translation of science into practice by generating crucial evidence for improving the uptake of effective health interventions. By design, they pose unique recruitment and retention challenges due to their aims, units of analysis, and sampling plans, which typically require many clinical sites (i.e. often 20 or more) and participation by individuals who are related across multiple levels (e.g. linked organizational leaders, clinicians, and patients). In this article, we present a new multilevel, theory-informed, and relationship-centered framework for conceptualizing recruitment and retention in implementation and hybrid effectiveness-implementation trials which integrates and builds on prior work on recruitment and retention strategies in patient-focused trials. We describe the framework's application in the Working to Implement and Sustain Digital Outcome Measures hybrid type III trial, which occurred in part during the COVID-19 pandemic.

methodsRecruitment for the Working to Implement and Sustain Digital Outcome Measures trial occurred from October 2019 to February 2022. Development of recruitment and retention strategies was guided by a newly developed multilevel framework, which targeted the capability, opportunity, and motivation of organizational leaders, clinicians, patient-facing administrative staff, and patients to engage in research. A structured assessment guide was developed and applied to refine recruitment and retention approaches throughout the trial. We describe the framework and its application amid the onset of the COVID-19 pandemic which required rapid adjustments to address numerous barriers.

resultsThe Working to Implement and Sustain Digital Outcome Measures trial enrolled 21 outpatient clinics in three US states, incorporating 252 clinicians and 686 caregivers of youth (95% of patient recruitment target) across two distinct phases. Data completion rates for organizational leaders and clinicians averaged 90% over five waves spanning 18 months, despite the onset of the COVID pandemic. Caregiver completion rates of monthly follow-up assessments ranged from 80%-88% across 6 months. This article presents the multilevel framework, assessment guide, and strategies used to achieve recruitment and retention targets at each level.

conclusionWe conducted a multi-state hybrid type III effectiveness-implementation trial that maintained high recruitment and retention across all relevant levels amid a global pandemic. The newly developed multilevel recruitment and retention framework and assessment guide presented here, which integrates behavioral theory, a relationship-focused lens, and evidence-based strategies for participant recruitment and retention at multiple levels, can be adapted and used by other researchers for implementation, hybrid, and multilevel pragmatic trials as well as other implementation studies.

Indexed as

Clinical Trials as TopicCOVID-19Patient SelectionHumansResearch DesignSARS-CoV-2children’s mental healthimplementation researchmultilevel designspragmatic trialsRecruitment and retention

Identifiers

PMID39797386
PMCPMC12094903

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.