Should ecologists prefer model- over distance-based multivariate methods?

Ecological data sets often record the abundance of species, together with a set of explanatory variables. Multivariate statistical methods are optimal to analyze such data and are thus frequently used in ecology for exploration, visualization, and inference. Most approaches are based on pairwise distance matrices instead of the sites-by-species matrix, which stands in stark contrast to univariate statistics, where data models, assuming specific distributions, are the norm. However, through advances in statistical theory and computational power, models for multivariate data have gained traction. Systematic simulation-based performance evaluations of these methods are important as guides for practitioners but still lacking. Here, we compare two model-based methods, multivariate generalized linear models (MvGLMs) and constrained quadratic ordination (CQO), with two distance-based methods, distance-based redundancy analysis (dbRDA) and canonical correspondence analysis (CCA). We studied the performance of the methods to discriminate between causal variables and noise variables for 190 simulated data sets covering different sample sizes and data distributions. MvGLM and dbRDA differentiated accurately between causal and noise variables. The former had the lowest false-positive rate (0.008), while the latter had the lowest false-negative rate (0.027). CQO and CCA had the highest false-negative rate (0.291) and false-positive rate (0.256), respectively, where these error rates were typically high for data sets with linear responses. Our study shows that both model- and distance-based methods have their place in the ecologist's statistical toolbox. MvGLM and dbRDA are reliable for analyzing species-environment relations, whereas both CQO and CCA exhibited considerable flaws, especially with linear environmental gradients.