Evidence map›Paper›PMID 36514059›Full record

SynthesisImplementation science : IS2022

Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review.

Alix Hall, Adam Shoesmith, Emma Doherty, Brydie McEvoy, Kayne Mettert, Cara C Lewis, Luke Wolfenden, Serene Yoong, Melanie Kingsland, Rachel C Shelton and 4 more

Abstract readSystematic Review
In one paragraph

Synthesis in Implementation science : IS, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 37 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
37citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

37 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Sustaining Diabetes Shared Medical Appointments After a Pragmatic Trial.Journal of the American Board of Family Medicine : JABFM
    Trial
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Review
  11. An introduction to implementation science in rehabilitation medicine.PM & R : the journal of injury, function, and rehabilitation · 2025
    Article
  12. Sustainability in translational genomics research with undiagnosed patients: What is it, why do we need it, and how do we do it?Genetics in medicine : official journal of the American College of Medical Genetics · 2025
    Article
  13. Article
  14. Article
  15. Review
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

14 authors.

Alix HallSchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia. alix.hall@newcastle.edu.au.ORCID 0000-0002-1043-6110
Adam ShoesmithSchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.
Emma DohertySchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.
Brydie McEvoySchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.
Kayne MettertKaiser Permanente Washington Health Research Institute, Seattle, USA.
Cara C LewisKaiser Permanente Washington Health Research Institute, Seattle, USA.
Luke WolfendenSchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.
Serene YoongSchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.
Melanie KingslandSchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.
Rachel C SheltonMailman School of Public Health, Department of Sociomedical Sciences, Columbia University, New York, New York, USA.
Shannon Wiltsey StirmanDissemination and Training Division, National Center for PTSD and Department of Psychiatry and Behavioural Sciences, Stanford Medicine, Stanford University, Palo Alto, California, USA.
Noor ImadHunter New England Population Health, Hunter New England Local Health District, Wallsend, NSW, Australia.
Rachel SutherlandSchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.
Nicole NathanSchool of Medicine and Public Health, The University of Newcastle, Locked Bag 10 Wallsend, Callaghan, NSW, Australia.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundSustainability is concerned with the long-term delivery and subsequent benefits of evidence-based interventions. To further this field, we require a strong understanding and thus measurement of sustainability and what impacts sustainability (i.e., sustainability determinants). This systematic review aimed to evaluate the quality and empirical application of measures of sustainability and sustainability determinants for use in clinical, public health, and community settings.

methodsSeven electronic databases, reference lists of relevant reviews, online repositories of implementation measures, and the grey literature were searched. Publications were included if they reported on the development, psychometric evaluation, or empirical use of a multi-item, quantitative measure of sustainability, or sustainability determinants. Eligibility was not restricted by language or date. Eligibility screening and data extraction were conducted independently by two members of the research team. Content coverage of each measure was assessed by mapping measure items to relevant constructs of sustainability and sustainability determinants. The pragmatic and psychometric properties of included measures was assessed using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The empirical use of each measure was descriptively analyzed.

resultsA total of 32,782 articles were screened from the database search, of which 37 were eligible. An additional 186 publications were identified from the grey literature search. The 223 included articles represented 28 individual measures, of which two assessed sustainability as an outcome, 25 covered sustainability determinants and one explicitly assessed both. The psychometric and pragmatic quality was variable, with PAPERS scores ranging from 14 to 35, out of a possible 56 points. The Provider Report of Sustainment Scale had the highest PAPERS score and measured sustainability as an outcome. The School-wide Universal Behaviour Sustainability Index-School Teams had the highest PAPERS score (score=29) of the measure of sustainability determinants.

conclusionsThis review can be used to guide selection of the most psychometrically robust, pragmatic, and relevant measure of sustainability and sustainability determinants. It also highlights that future research is needed to improve the psychometric and pragmatic quality of current measures in this field.

trial registrationThis review was prospectively registered with Research Registry (reviewregistry1097), March 2021.

Indexed as

Public HealthHumansPsychometricsDeterminantsMeasurementPsychometricsSustainability

Identifiers

PMID36514059
PMCPMC9746194

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.