Evidence map›Paper›PMID 31473029›Full record

ReviewPsychiatry research2020

Quantitative approaches for the evaluation of implementation research studies.

Justin D Smith, Mohamed Hasan

Registry-linked trialAbstract readReview
In one paragraph

Review in Psychiatry research, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT04321876 (Evaluation of a Collaborative Behavioral Health Program Implementation in Primary Care), which is not on this map. Cited by 37 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
37citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT04321876 active not recruitingnot on this map

Evaluation of a Collaborative Behavioral Health Program Implementation in Primary Care

Typeobservational_patient_registrySponsorNorthwestern UniversityRan2018 to 2026Enrolled778ConditionsDepressionArmsCollaborative Behavioral Health Program
3 · Its place in the literature

Who cites it

37 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Trial
  4. Trial
  5. Trial
  6. Trial
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. An innovative multimorbidity patient-centered care model in Chile: implementation evaluation results.Archives of public health = Archives belges de sante publique · 2025
    Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

2 authors.

Justin D SmithCenter for Prevention Implementation Methodology (Ce-PIM) for Drug Abuse and HIV, Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, 750 N Lake Shore Dr., Chicago, IL, USA. Electronic address: jd.smith@northwestern.edu.
Mohamed HasanCenter for Healthcare Studies, Institute of Public Health and Medicine, Northwestern University Feinberg School of Medicine, 633 N St. Claire St., Chicago, Illinois, USA. Electronic address: mohamedhasan2020@u.northwestern.edu.

Funding

INNOVATIVE IMPLEMENTATION RESEARCH METHODS COREP30DA027828 · NIDA · UNIVERSITY OF MIAMI SCHOOL OF MEDICINE · PI BROWN, C. HENDRICKS, MUSTANSKI, BRIAN · 2011 to 2020
$11.4M
NIDA NIH HHS P30 DA027828
6 · The paper itself

Abstract

Implementation research necessitates a shift from clinical trial methods in both the conduct of the study and in the way that it is evaluated given the focus on the impact of implementation strategies. That is, the methods or techniques to support the adoption and delivery of a clinical or preventive intervention, program, or policy. As strategies target one or more levels within the service delivery system, evaluating their impact needs to follow suit. This article discusses the methods and practices involved in quantitative evaluations of implementation research studies. We focus on evaluation methods that characterize and quantify the overall impacts of an implementation strategy on various outcomes. This article discusses available measurement methods for common quantitative implementation outcomes involved in such an evaluation-adoption, fidelity, implementation cost, reach, and sustainment-and the sources of such data for these metrics using established taxonomies and frameworks. Last, we present an example of a quantitative evaluation from an ongoing randomized rollout implementation trial of the Collaborative Care Model for depression management in a large primary healthcare system.

Indexed as

Evaluation Studies as TopicImplementation ScienceBiomedical ResearchDelivery of Health CareDepressionHumansIntersectoral CollaborationPrimary Health CareProgram EvaluationResearch DesignImplementation measurementImplementation researchSummative evaluation

Identifiers

PMID31473029
PMCPMC7176071

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.