Variable selection strategies for genomic prediction of growth and carcass related traits in experimental Nellore cattle herds under different selection criteria.

Genomic selection (GS) has become a widely used tool in breeding programs, enhancing selection accuracy and leading to faster genetic progress. However, in small populations, GS faces challenges due to limited data and a large number of markers potentially leading to biased predictions. Implementing feature selection strategies is essential to improve prediction accuracy and avoid overfitting. Hence, we compared the predictive ability of genomic best linear unbiased prediction (GBLUP), Bayesian B (BayesB), and elastic net (ENet) models, using all markers and feature selection via GWAS and fixation index (FST) to reduce marker numbers, for growth and ultrasound carcass traits in three Nellore cattle populations differentially selected for yearling body weight (YBW). The populations evaluated included: Nellore Control (NeC), selected for YBW; Nellore Selection (NeS), selected for maximum YBW; and Nellore Traditional (NeT), selected for maximum YBW and lower residual feed intake (RFI) since 2013. Comparing the statistical approaches using GBLUP as the reference, ENet improved prediction accuracy by 10% for growth traits and 12% for carcass traits, while BayesB showed no improvement for growth traits but achieved a 3% gain for carcass traits. When comparing models using all markers to those with variable selection, both GWAS and FST improved prediction accuracy across models, with FST outperforming GWAS in stratified populations. A stricter GWAS threshold (> 1.0% explained variance), compared to a less conservative criterion (> 0.5%), reduced BayesB prediction accuracy (6.8%), while slightly increasing accuracy for GBLUP (1.3%) and ENet (2.4%). Similarly, a more restrictive FST threshold (> 0.2) against a less conservative (> 0.1) resulted in smaller gains for GBLUP (4%) and ENet (5%), but reduced BayesB accuracy (- 4%). Overall, selecting markers through GWAS and FST improves prediction accuracy for both growth and carcass traits, particularly in stratified populations. However, stricter thresholds can negatively impact accuracy, highlighting the need for optimized marker selection strategies.