Validating Physiologically-Based Pharmacokinetic Models Using the Continuous Ranked Probability Score: Beyond Being Correct on Average.

Physiologically-based pharmacokinetic (PBPK) models have become increasingly popular for model-informed drug development (MIDD) over the past decade. While several guidelines for model evaluation exist, these are by design often of a general and non-specific nature. It is clear what steps should be carried out but not necessarily how. For instance, a validation step needs to be performed to check if the predictions of a model indeed match with an external dataset. However, how to quantify this is yet unspecified. In this paper, we propose a more thorough validation approach based on the Continuous Ranked Probability Score (CRPS), a popular metric that explicitly quantifies how well a model recreates the distribution of observed data. Crucially, when applied to PBPK modeling, this metric can be used both in situations where we have individual level predictions and in situations where an entire virtual population is created. The CRPS can also be used to quantify the difference in predictive performance of two competing models. We applied this validation technique to compare two PBPK models. Additionally, we show that using a skill-score approach facilitates the validation of a single model. While our paper focuses on PBPK models, this metric is equally applicable to other models where the goal is to create a virtual population. Additionally, we provide an easily accessible online tool that can be used to perform the proposed validation method.