Validation of neuron activation patterns for artificial intelligence models in oculomics.

Recent advancements in artificial intelligence (AI) have prompted researchers to expand into the field of oculomics; the association between the retina and systemic health. Unlike conventional AI models trained on well-recognized retinal features, the retinal phenotypes that most oculomics models use are more subtle. Consequently, applying conventional tools, such as saliency maps, to understand how oculomics models arrive at their inference is problematic and open to bias. We hypothesized that neuron activation patterns (NAPs) could be an alternative way to interpret oculomics models, but currently, most existing implementations focus on failure diagnosis. In this study, we designed a novel NAP framework to interpret an oculomics model. We then applied our framework to an AI model predicting systolic blood pressure from fundus images in the United Kingdom Biobank dataset. We found that the NAP generated from our framework was correlated to the clinically relevant endpoint of cardiovascular risk. Our NAP was also able to discern two biologically distinct groups among participants who were assigned the same predicted systolic blood pressure. These results demonstrate the feasibility of our proposed NAP framework for gaining deeper insights into the functioning of oculomics models. Further work is required to validate these results on external datasets.