ArticleImplementation science communications2025
Considerations for evaluating pragmatic design elements in digital health intervention trials: the case of Keep It Up! 3.0.
Article in Implementation science communications, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 2 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
2 citing papers in PubMed.
- When scientific constraints and real-world implementation collide: Lessons from a hybrid type 3 study of a digital, direct-to-consumer HIV prevention program.Research square · 2026Article
- Incentivizing Prevention: Community Based Organizations' Perceptions on Financial Incentives for Recruitment and Retention of Participants in Keep it Up! (KIU!), an eHealth HIV Prevention Intervention.AIDS and behavior · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
Abstract
backgroundDigital health interventions are increasingly promoted in healthcare and prevention practices due to their potential for reaching key populations in a cost-efficient manner. Yet there has been limited research on how to effectively implement them with pragmatic approaches that can facilitate scale-up. Keep It Up! (KIU!) 3.0 was a hybrid type 3 implementation-effectiveness trial comparing two delivery strategies (i.e. trial arms) of an HIV prevention intervention for cisgender, young men who have sex with men. We aimed to determine the level of pragmatism of our two-armed trial before and after changes to the county-randomized design.
methodsWe applied different versions of the PRagmatic Explanatory Continuum Indicator Summary (PRECIS) tool to the two trial arms: delivery of KIU! by community-based organizations (CBO) versus centralized, direct-to-consumer (DTC) delivery. We scored PRECIS-2 for the original study design and the modified design in which the DTC strategy expanded nationally. We applied PRECIS-2-PS to the modified study design. Nine coders in three groups independently scored the tools. Scores were iteratively discussed to arrive at one consensus score per domain, tool, design stage, and arm. We plotted results using the PRECIS-2 and PRECIS-2-PS wheels and averaged domains scores to describe overall score along the Pragmatic-Explanatory Continuum.
resultsUsing PRECIS-2, the trial was on the pragmatic side of the spectrum for both arms and design stages, with average ratings ranging from 3.89-4.33. Both arms were highly pragmatic in the original and modified design in the Setting and Primary Analysis domains and least pragmatic in the Follow-up domain. In the modified trial design, the CBO and DTC arms again scored rather pragmatic using the PRECIS-2-PS tool, but CBO arm scored higher in the eligibility, recruitment, and organization domains compared to PRECIS-2 (5 vs. 4, respectively).
conclusionsApplication of both the PRECIS-2 and PRECIS-2-PS tools validated the pragmatic design of KIU! 3.0 as originally designed and after modifications during trial implementation. Our findings highlight instances where one tool may be more suitable than the other to assess the pragmatic-explanatory continuum for emerging digital health interventions delivered in diverse settings and with different implementation strategies.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.