ArticleTranslational behavioral medicine2021
Pragmatic measures for implementation research: development of the Psychometric and Pragmatic Evidence Rating Scale (PAPERS).
Article in Translational behavioral medicine, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 56 papers, 6 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
56 citing papers in PubMed, 6 syntheses or guidelines pooled it.
- Systematic review of patient-reported outcome measures for opioid use disorder recovery.Addiction (Abingdon, England) · 2026Pooled it
- Tools for assessing health research partnership outcomes and impacts: a systematic review.Health research policy and systems · 2023Pooled it
- How are health research partnerships assessed? A systematic review of outcomes, impacts, terminology and the use of theories, models and frameworks.Health research policy and systems · 2022Pooled it
- Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review.Implementation science : IS · 2022Pooled it
- Psychometric properties of leadership scales for health professionals: a systematic review.Implementation science : IS · 2021Pooled it
- Quantitative measures of health policy implementation determinants and outcomes: a systematic review.Implementation science : IS · 2020Pooled it
- Scoping review of instruments measuring organizational context in implementation studies within healthcare.Health research policy and systems · 2026Review
- Evaluation of the implementation of a mobility plan in a geriatric clinic in Switzerland - a quality improvement study.BMC geriatrics · 2026Article
- The See It to Stop It Indicator Tool to Identify Child Sex Trafficking: Acceptability, Feasibility, Utility, and Outcomes.Journal of child sexual abuse · 2026Article
- Self-report methods for the assessment of implementation determinants: Reflections on the CFIR user guide.Implementation science : IS · 2026Article
- A Measurement Roadmap to Guide Implementation Practitioners in Selecting Implementation Measures.Global implementation research and applications · 2025Article
- The Impact of Drug Delivery Systems on Pharmacokinetics and Drug-Drug Interactions in Neuropsychiatric Treatment.Cureus · 2025Review
- The role of organizational characteristics in intervention sustainment: findings from a quantitative analysis in 42 HIV testing clinics in Vietnam.Implementation science communications · 2025Article
- Expanding the pragmatic lens in implementation science: why stakeholder perspectives matter.Implementation science communications · 2025Article
- Developing a new pragmatic tool for assessing contextual fit and feasibility of evidence-based interventions towards effective implementation in global health.BMJ global health · 2025Article
- Developing theoretically grounded strategies to enable and promote patient and public involvement in implementation research studies.Research involvement and engagement · 2025Article
- Development and validation of a pragmatic measure of cocreation in research engagement: a study protocol.BMJ open · 2024Article
- Evaluation of a Pilot College Student-Based Physical Activity Implementation Strategy in a Rural Middle School.International journal of environmental research and public health · 2024Article
- A case study of the development of a valid and pragmatic implementation science measure: the Barriers and Facilitators in Implementation of Task-Sharing Mental Health interventions (BeFITS-MH) measure.BMC health services research · 2024Article
- Lowering the burden: Shorter versions of the Program Sustainability Assessment Tool (PSAT) and Clinical Sustainability Assessment Tool (CSAT).Implementation science communications · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
Abstract
The use of reliable, valid measures in implementation practice will remain limited without pragmatic measures. Previous research identified the need for pragmatic measures, though the characteristic identification used only expert opinion and literature review. Our team completed four studies to develop a stakeholder-driven pragmatic rating criteria for implementation measures. We published Studies 1 (identifying dimensions of the pragmatic construct) and 2 (clarifying the internal structure) that engaged stakeholders-participants in mental health provider and implementation settings-to identify 17 terms/phrases across four categories: Useful, Compatible, Acceptable, and Easy. This paper presents Studies 3 and 4: a Delphi to ascertain stakeholder-prioritized dimensions within a mental health context, and a pilot study applying the rating criteria. Stakeholders (N = 26) participated in a Delphi and rated the relevance of 17 terms/phrases to the pragmatic construct. The investigator team further defined and shortened the list, which were piloted with 60 implementation measures. The Delphi confirmed the importance of all pragmatic criteria, but provided little guidance on relative importance. The investigators removed or combined terms/phrases to obtain 11 criteria. The 6-point rating system assigned to each criterion demonstrated sufficient variability across items. The grey literature did not add critical information. This work produced the first stakeholder-driven rating criteria to assess whether measures are pragmatic. The Psychometric and Pragmatic Evidence Rating Scale (PAPERS) combines the pragmatic criteria with psychometric rating criteria, from previous work. Use of PAPERS can inform development of implementation measures and to assess the quality of existing measures.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.