Detecting steps in spatial genetic data: Which diversity measures are best?

Accurately detecting sudden changes, or steps, in genetic diversity across landscapes is important for locating barriers to gene flow, identifying selectively important loci, and defining management units. However, there are many metrics that researchers could use to detect steps and little information on which might be the most robust. Our study aimed to determine the best measure/s for genetic step detection along linear gradients using biallelic single nucleotide polymorphism (SNP) data. We tested the ability to differentiate between linear and step-like gradients in genetic diversity, using a range of diversity measures derived from the q-profile, including allelic richness, Shannon Information, GST, and Jost-D, as well as Bray-Curtis dissimilarity. To determine the properties of each measure, we repeated simulations of different intensities of step and allele proportion ranges, with varying genome sample size, number of loci, and number of localities. We found that alpha diversity (within-locality) based measures were ineffective at detecting steps. Further, allelic richness-based beta (between-locality) measures (e.g., Jaccard and Sørensen dissimilarity) were not reliable for detecting steps, but instead detected departures from fixation. The beta diversity measures best able to detect steps were: Shannon Information based measures, GST based measures, a Jost-D related measure, and Bray-Curtis dissimilarity. No one measure was best overall, with a trade-off between those measures with high step detection sensitivity (GST and Bray-Curtis) and those that minimised false positives (a variant of Shannon Information). Therefore, when detecting steps, we recommend understanding the differences between measures and using a combination of approaches.