Analysis of rat toxicology studies: statistical agreement between virtual and concurrent controls in detecting treatment effects on liver enzymes.

Introduction: Virtual control groups (VCGs) are proposed as a means to reduce reliance on live animals in preclinical toxicology studies. Despite the potential benefits of VCGs, concerns remain regarding their comparability to traditional concurrent control groups (CCGs), and the risk of introducing bias into preclinical toxicology research. Our simulation focused on observing the statistical consequences of replacing the CCG (fully or partially) with VCGs derived from historical data, assessing their feasibility and limitations, and determining if VCG use alters study conclusions on selected toxicological endpoints. Methods: We conducted simulations comparing liver enzyme activity in studies using VCGs and those using CCGs. VCGs were generated by simple matching using a limited set of criteria to a large-scale dataset. One hundred VCGs were generated for each reanalyzed study and the percentage of agreement with CCGs was then assessed across a variety of statistical measures for a selection of endpoints, i.e., liver enzymes and bodyweight. Results: Full agreement was observed in 46.9% of the performed comparisons between VCGs and CCGs, while full disagreement was noted in 2.6% of comparisons. For the remaining instances of partial agreement, the sampled VCGs predominantly aligned with the CCGs. Discrepancies observed did not consistently affect all measured endpoints within a given analyzed study; they were especially noted when CCG enzyme activity levels approached the boundaries of reference intervals, differing significantly from the majority of the other controls in the database. Partial replacement of CCG with VCG improved the overall agreement. The likelihood of reaching a different conclusion for the overall study outcome was considered to be low. Among the statistical methods used for comparing controls versus treated animals, the method based on the calculation of effect size appeared to be the most appropriate to use, especially with our resampling approach. Discussion: Our results suggest VCGs could be a potential alternative to CCGs. Key next steps that will need to be addressed include enhancing datasets with richer metadata and standardizing variables across studies to facilitate the selection of matching criteria, refining methods for managing outliers and hypothesis testing, and establishing proxy variables for animal growth.