Evidence map›Paper›PMID 27745684›Full record

ArticleAmerican journal of preventive medicine2016

Evaluating Digital Health Interventions: Key Questions and Approaches.

Elizabeth Murray, Eric B Hekler, Gerhard Andersson, Linda M Collins, Aiden Doherty, Chris Hollis, Daniel E Rivera, Robert West, Jeremy C Wyatt

Registry-linked trialAbstract readEditorial
PubMed Full text at PMC Publisher
In one paragraph

Article in American journal of preventive medicine, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT07561957 (A Smart Phone Application to Improve Adoption of the 2024 KDIGO CKD Guidelines), which is not on this map. Cited by 498 papers, 30 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
498citing papers in PubMed, 30 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT07561957 narecruitingnot on this mapstarted 2026, after this paper: background citation

A Smart Phone Application to Improve Adoption of the 2024 KDIGO CKD Guidelines

TypeinterventionalSponsorSt. James's Hospital, IrelandRan2026 to 2027Enrolled80ConditionsChronic Kidney Disease, Proteinuria, Blood Pressure Control, Congenital Anomalies of the Kidneys and Urinary TractArmsUse of an electronic digital Application
3 · Its place in the literature

Who cites it

498 citing papers in PubMed, 30 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

438 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors.

Elizabeth MurrayResearch Department of Primary Care and Population Health, University College London, London, United Kingdom. Electronic address: elizabeth.murray@ucl.ac.uk.
Eric B HeklerDesigning Health Lab, School of Nutrition and Health Promotion, Arizona State University, Phoenix, Arizona.
Gerhard AnderssonDepartment of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden; Department of Clinical Neuroscience, Karolinska Institute, Stockholm, Sweden.
Linda M CollinsThe Methodology Center and Department of Human Development and Family Studies, The Pennsylvania State University, State College, Pennsylvania.
Aiden DohertyMRC Clinical Trial Service Unit Hub, Nuffield Department of Population Health, University of Oxford, Oxford, United Kingdom.
Chris HollisNIHR MindTech HTC, University of Nottingham, Nottingham, United Kingdom.
Daniel E RiveraSchool for the Engineering of Matter, Transport, and Energy, Ira A. Fulton Schools of Engineering, Arizona State University, Phoenix, Arizona.
Robert WestResearch Department of Epidemiology and Public Health, University College London, London, United Kingdom.
Jeremy C WyattWessex Institute, University of Southampton, Southampton, United Kingdom.

Funding

Pilot, Mentoring, and Professional Development CoreP50DA039838 · NIDA · PENNSYLVANIA STATE UNIVERSITY, THE · PI LI, RUNZE · 2015 to 2019
$13.9M
British Heart Foundation RE/13/1/30181Cancer Research UK 14135Department of Health RP-PG-0609-10135Medical Research Council MR/L004933/1Medical Research Council MR/L004933/2NIDA NIH HHS P50 DA039838
6 · The paper itself

Abstract

Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research.

Indexed as

Health BehaviorHealth PromotionProgram EvaluationResearch DesignTelecommunicationsHumans

Identifiers

PMID27745684
PMCPMC5324832

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.