Removing scanner effects with a multivariate latent approach: A RELIEF for the ABCD imaging data?

Scan site harmonization is a crucial part of any neuroimaging analysis when data have been pooled across different study sites. Zhang and colleagues recently introduced the multivariate harmonization method RELIEF (REmoval of Latent Inter-scanner Effects through Factorization), aiming to remove explicit and latent scan site effects. Their initial validation in an adult sample showed superior performance compared to established methods. We here sought to investigate utility of RELIEF in harmonizing data from the Adolescent Brain and Cognitive Development (ABCD) study, a widely used resource for developmental brain imaging. We benchmarked RELIEF against unharmonized, ComBat, and CovBat harmonized data and investigated the impact of manufacturer type, sample size, and a narrow sample age range on harmonization performance. We found that in cases where sites with sufficiently large samples were harmonized, RELIEF outperformed other techniques, yet in cases where sites with very small samples were included there was substantial performance variation unique to RELIEF. Our results therefore highlight the need for careful quality control when harmonizing data sets with imbalanced samples like the ABCD cohort. Our comment alongside shared scripts may provide guidance for other scholars wanting to integrate best practices in their ABCD related work.