To Collapse or not to Collapse Multiple Measures of the Same Psychological Construct: A Demonstration Across 14 Independent Studies of Autistic and Non-Autistic Children.

Large data repositories provide a valuable opportunity to combine multiple independent studies, creating large samples that can address questions not achievable within a single study. However, independent studies often use different instruments to measure the same behavior or psychological construct, such as cognitive ability, social functioning, or psychopathology. Investigators often combine samples that include different instruments under the assumption that instruments assessing the same construct capture the same underlying phenomenon. Yet this assumption of instrument interchangeability may be wrong. Collapsing different instruments into a single variable could introduce bias. Here, we established a quantitative method to demonstrate compatibility among multiple instruments with moderation analyses. We illustrate this method in domains of cognitive ability, anxiety symptoms, and attention-deficit/hyperactivity disorder symptoms as exemplars that might interact with outcomes of interest (autistic features instruments). A total of 1,352 participants were included from autistic and non-autistic samples across 14 independent studies. Our findings suggest that within this sample, some commonly used instruments within a domain can be acceptably combined, whereas others demonstrate discrepant relationships with outcomes of interest. More importantly, we offer a generalizable approach for scientists seeking to leverage existing datasets for secondary analyses.