Evidence map›Paper›PMID 40410922›Full record

ArticleImplementation science communications2025

Using a novel, multi-method approach to evaluate the quality of self-report survey data in organizational readiness research.

Derek W Craig, Jocelyn Hunyadi, Timothy J Walker, Lauren Workman, Maria McClam, Andrea Lamont, Joe R Padilla, Pamela Diamond, Abraham Wandersman, Maria E Fernandez

Abstract read
In one paragraph

Article in Implementation science communications, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.

0numbers the graph read from it
0cells of the map it votes in
1citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

1 citing paper in PubMed.

  1. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Derek W CraigCenter for Health Promotion and Prevention Research, UTHealth Houston School of Public Health, Houston, Texas, USA. Derek.W.Craig@uth.tmc.edu.ORCID http://orcid.org/0000-0003-3589-0755
Jocelyn HunyadiCenter for Spatial-Temporal Modeling for Applications in Population Sciences, UTHealth Houston School of Public Health, Houston, Texas, USA.
Timothy J WalkerCenter for Health Promotion and Prevention Research, UTHealth Houston School of Public Health, Houston, Texas, USA.
Lauren WorkmanArnold School of Public Health, University of South Carolina, Columbia, South Carolina, USA.
Maria McClamArnold School of Public Health, University of South Carolina, Columbia, South Carolina, USA.
Andrea LamontWandersman Center, Columbia, South Carolina, USA.
Joe R PadillaCenter for Health Promotion and Prevention Research, UTHealth Houston School of Public Health, Houston, Texas, USA.
Pamela DiamondCenter for Health Promotion and Prevention Research, UTHealth Houston School of Public Health, Houston, Texas, USA.
Abraham WandersmanWandersman Center, Columbia, South Carolina, USA.
Maria E FernandezCenter for Health Promotion and Prevention Research, UTHealth Houston School of Public Health, Houston, Texas, USA.

Funding

Cancer Prevention and Control Research Training and Career Development ProgramT32CA057712 · NCI · UNIVERSITY OF TEXAS HLTH SCI CTR HOUSTON · PI Bijal A. Balasubramanian, Maria Eulalia Fernandez · 2018 to 2026
$3.2M
Development and Validation of a Measure of Organizational Readiness (motivation x capacity) for ImplementationR01CA228527 · NCI · UNIVERSITY OF TEXAS HLTH SCI CTR HOUSTON · PI FERNANDEZ, MARIA EULALIA · 2019 to 2023
$3.1M
Developing And Testing An Implementation Strategy For Active Learning To Promote Physical Activity In ChildrenK01HL151817 · NHLBI · UNIVERSITY OF TEXAS HLTH SCI CTR HOUSTON · PI WALKER, TIMOTHY JAMES · 2021 to 2024
$567k
NCI NIH HHS 1R01CA228527-05NCI NIH HHS R01 CA228527NCI NIH HHS T32 CA057712NCI NIH HHS T32/CA057712NHLBI NIH HHS K01 HL151817NHLBI NIH HHS K01HL151817
6 · The paper itself

Abstract

backgroundSelf-report measures are essential in implementation science since many phenomena are difficult to assess directly. Nevertheless, cognitively demanding surveys increase the prevalence of careless and inattentive responses. Assessing response quality is critical to improving data validity, yet recommendations for determining response quality vary. To address this, we aimed to 1) apply a multi-method approach to assess the quality of self-report survey data in a study aimed to validate a measure of organizational readiness, 2) compare readiness scores among responses categorized as high- and low-quality, and 3) examine individual characteristics associated with low-quality responses.

methodsWe surveyed federally qualified health center staff to assess organizational readiness for implementing evidence-based interventions to increase colorectal cancer screening. The survey was informed by the R = MC

resultsThe sample consisted of 474 responses from 57 clinics. The average survey time was 24.3 min, and 42 respondents (8.9%) had MRPs on all readiness components. The number of low-quality responses varied by assessment method (range = 42-98). Survey responses classified as low quality had higher readiness scores (M, ISC, GC, p < 0.01). Age (p = 0.01), race (p < 0.01), and implementation involvement (p = 0.04) were inversely associated with survey completion time, whereas older age (p = 0.01) and more years worked at the clinic (p = 0.03) were associated with higher response quality. Quality improvement staff and clinic management were less likely to provide low-quality responses (p = 0.04).

conclusionsOur findings suggest that readiness scores can be inflated by low-quality responses, and individual characteristics play a significant role in data quality. There is a need to be aware of who is completing surveys and the context in which surveys are distributed to improve the interpretation of findings and make the measurement of implementation-related constructs more precise.

Indexed as

Data qualityMeasure developmentOrganizational readinessSurvey research methods

Identifiers

PMID40410922
PMCPMC12102923

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.