ReviewBehavior research methods2023
Psychometric properties of sum scores and factor scores differ even when their correlation is 0.98: A response to Widaman and Revelle.
Review in Behavior research methods, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed.
- Addressing Methodological Challenges in Follow-Up RCTs During the COVID-19 Pandemic: The Impact of the Good Behavior Game and MyTeachingPartner™ on Teacher Burnout and Self-Efficacy.Prevention science : the official journal of the Society for Prevention Research · 2026Trial
- Beyond Jacobson and Truax: Estimation of Clinical Significance Trajectories in the Coping Power Intervention Using Measurement Error-Corrected Multilevel Modeling.Behavior therapy · 2025Trial
- Improving the comparability of momentary affect assessments and accounting for item-specific effects in within-person affect dynamics.Communications psychology · 2026Article
- Mitigating the slipping effect in polytomous scales: The Generalized Conditional Reliability Weighting (G-CRW) Algorithm and the WeightMyItems R package.Behavior research methods · 2026Article
- Factor Scores in Small Samples: Recommendations and Solutions.Educational and psychological measurement · 2026Article
- Identifying Neurodevelopmental Domain Subgroups in Autism and ADHD.Journal of autism and developmental disorders · 2026Article
- Psychometric modeling of Boston process approach data for dementia prediction in the Framingham Heart Study.Journal of the International Neuropsychological Society : JINS · 2026Article
- The individual-level precision of implicit measures.Behavior research methods · 2025Article
- The Italian Version of the Difficulties in Emotion Regulation Scale-8 (DERS-8): A Two-Step Assessment of Structural Validity, Psychometric Properties, and Clinical Cut-Off.BMC psychology · 2025Article
- Applying Bayesian checks of cancellation axioms for interval scaling in limited samples.Behavior research methods · 2025Article
- The seven deadly sins: measuring overvaluation of social media with the Plan-net 25 scale.BMC psychology · 2025Article
- What you see is not what you get: Observed scale score comparisons misestimate true group differences.Behavior research methods · 2025Article
- Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution?Behavior research methods · 2025Article
- Deriving Expected Values of Model Parameters when Using Sum Scores in Simulation Research.Structural equation modeling : a multidisciplinary journal · 2025Article
- Practical Implications of Sum Scores Being Psychometrics' Greatest Accomplishment.Psychometrika · 2024Article
- Psychometric evaluation of the Adelphi Adherence Questionnaire (ADAQ©) in adults with osteoarthritis.Journal of patient-reported outcomes · 2024Article
- Exploring the genetic etiology across the continuum of the general psychopathology factor: a Swedish population-based family and twin study.Molecular psychiatry · 2024Article
- The Hidden Psychometric Model Underlying thePsychiatric services (Washington, D.C.) · 2024Article
- Thinking About Sum Scores Yet Again, Maybe the Last Time, We Don't Know, Oh No . . .: A Comment on.Educational and psychological measurement · 2024Article
- Empathy and Coping Strategies Predict Quality of Life in Japanese Healthcare Professionals.Behavioral sciences (Basel, Switzerland) · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Commentary in Widaman and Revelle (2022) argued that sum scoring is justified as long as unidimensionality holds because sum score reliability is defined. My response begins with a review of the literature supporting the perspective we adopted in the original article. I then conduct simulation studies to assess the psychometric properties of sum scores created using Widaman and Revelle's justification relative to scores created by the weighted factor score approach in the original article. In my simulations, I generate data where sum and factor scores are correlated at 0.96 or 0.98 because high factor-sum score correlations are often used to support the contention that sum and factor scores have interchangeable psychometric properties. I explore (a) correlations between estimated scores and true scores, (b) classification accuracy of sum and factor scores, and (c) reliability of sum and factor scores. Results show that factor scores have (a) higher correlations with true scores (Δ = 0.02-0.04), (b) higher sensitivity (Δ = 4-8 percentage points), and (c) higher reliability (Δ = 0.04-0.07). Factor score performance metrics also have less sampling variability in most conditions. Psychometric properties of sum scores-even when highly correlated with factor scores-remain less desirable than those of factor scores. Additional considerations like models with multiple factors and measurement invariance are also discussed. Essentially, even if accepting Widaman and Revelle's justification for sum scoring, it is uncertain whether researchers generally would want to sum score after fitting a factor analysis unless sum and factor scores correlate at (and not merely close to) 1.00.
Indexed as
Identifiers
36394821What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.