Evidence map›Paper›PMID 29695295›Full record

ArticleSystematic reviews2018

An updated protocol for a systematic review of implementation-related measures.

Cara C Lewis, Kayne D Mettert, Caitlin N Dorsey, Ruben G Martinez, Bryan J Weiner, Elspeth Nolen, Cameo Stanick, Heather Halko, Byron J Powell

Abstract read
In one paragraph

Article in Systematic reviews, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 52 papers, 7 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
52citing papers in PubMed, 7 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

52 citing papers in PubMed, 7 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Trial
  9. Article
  10. An introduction to implementation science in rehabilitation medicine.PM & R : the journal of injury, function, and rehabilitation · 2025
    Article
  11. Article
  12. Implementation Mapping to Identify Strategies to Increase Timely Postoperative Radiotherapy Initiation for Head/Neck Cancer.Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery · 2025
    Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. The snapshot audit methodology: design, implementation and analysis of prospective observational cohort studies in surgery.European journal of trauma and emergency surgery : official publication of the European Trauma Society · 2023
    Review
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors.

Cara C LewisKaiser Permanente Washington Health Research Institute, MacColl Center for Health Care Innovation, 1730 Minor Avenue, Suite 1600, Seattle, WA, 98101, USA. lewis.cc@ghc.org.
Kayne D MettertKaiser Permanente Washington Health Research Institute, MacColl Center for Health Care Innovation, 1730 Minor Avenue, Suite 1600, Seattle, WA, 98101, USA.
Caitlin N DorseyKaiser Permanente Washington Health Research Institute, MacColl Center for Health Care Innovation, 1730 Minor Avenue, Suite 1600, Seattle, WA, 98101, USA.
Ruben G MartinezPsychology Department, Virginia Commonwealth University, 806 W. Franklin St, Box 842018, Richmond, VA, 23284, USA.
Bryan J WeinerDepartment of Global Health, University of Washington, 1510 San Juan Road, Box 357965, Seattle, WA, 98195, USA.
Elspeth NolenDepartment of Global Health, University of Washington, 1510 San Juan Road, Box 357965, Seattle, WA, 98195, USA.
Cameo StanickHathaway-Sycamores Child and Family Services, 210 S DeLacey Ave, Suite 110, Pasadena, CA, 91105-2074, USA.
Heather HalkoDepartment of Psychology, University of Montana, 32 Campus Drive, Missoula, MT, 59812, USA.
Byron J PowellDepartment of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill, NC, 27599, USA.

Funding

Advancing Implementation Science through Measure Development and EvaluationR01MH106510 · NIMH · TRUSTEES OF INDIANA UNIVERSITY · PI LEWIS, CARA CHARISSA · 2015 to 2017
$1.3M
IMPROVING THE IMPLEMENTATION AND SUSTAINMENT OF EBPS IN MENTAL HEALTH: DEVELOPING AND PILOTING THE COLLABORATIVE ORGANIZATIONAL APPROACH TO SELECTING AND TAILORING IMPLEMENTATION STRATEGIES (COAST-ISK01MH113806 · NIMH · WASHINGTON UNIVERSITY · PI POWELL, BYRON JAMES · 2017 to 2020
$697k
NIMH NIH HHS R01 MH106510NIMH NIH HHS R01MH106510
6 · The paper itself

Abstract

backgroundImplementation science is the study of strategies used to integrate evidence-based practices into real-world settings (Eccles and Mittman, Implement Sci. 1(1):1, 2006). Central to the identification of replicable, feasible, and effective implementation strategies is the ability to assess the impact of contextual constructs and intervention characteristics that may influence implementation, but several measurement issues make this work quite difficult. For instance, it is unclear which constructs have no measures and which measures have any evidence of psychometric properties like reliability and validity. As part of a larger set of studies to advance implementation science measurement (Lewis et al., Implement Sci. 10:102, 2015), we will complete systematic reviews of measures that map onto the Consolidated Framework for Implementation Research (Damschroder et al., Implement Sci. 4:50, 2009) and the Implementation Outcomes Framework (Proctor et al., Adm Policy Ment Health. 38(2):65-76, 2011), the protocol for which is described in this manuscript.

methodsOur primary databases will be PubMed and Embase. Our search strings will be comprised of five levels: (1) the outcome or construct term; (2) terms for measure; (3) terms for evidence-based practice; (4) terms for implementation; and (5) terms for mental health. Two trained research specialists will independently review all titles and abstracts followed by full-text review for inclusion. The research specialists will then conduct measure-forward searches using the "cited by" function to identify all published empirical studies using each measure. The measure and associated publications will be compiled in a packet for data extraction. Data relevant to our Psychometric and Pragmatic Evidence Rating Scale (PAPERS) will be independently extracted and then rated using a worst score counts methodology reflecting "poor" to "excellent" evidence. DISCUSSION: We will build a centralized, accessible, searchable repository through which researchers, practitioners, and other stakeholders can identify psychometrically and pragmatically strong measures of implementation contexts, processes, and outcomes. By facilitating the employment of psychometrically and pragmatically strong measures identified through this systematic review, the repository would enhance the cumulativeness, reproducibility, and applicability of research findings in the rapidly growing field of implementation science.

Indexed as

Evidence-Based PracticeHealth Plan ImplementationSystematic Reviews as TopicHumansDisseminationEvidence-based assessmentImplementationInstrumentsMeasuresPragmaticPsychometricsSystematic review

Identifiers

PMID29695295
PMCPMC5918558

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.