Counting cells can accurately predict small-molecule bioactivity benchmarks.

Accurately predicting the activity of a chemical in each bioactivity assay based on its already known properties is extremely useful in drug development. Unfortunately, we discovered that many assays in widely used assay-activity benchmark datasets directly relate to cell health and cytotoxicity. Many other assays intend to capture a more specific phenotype, but their active compounds impact cell count, while inactives do not. In both cases, counting cells achieves unexpectedly high performance in these benchmarks, making them less useful for discerning whether additional properties, such as phenotypic profiles (mRNA or Cell Painting), provide additional useful information on bioactivity. To accomplish this goal, we recommend filtering benchmarks to exclude such assays and including a cell-count baseline. Using a benchmark with 24 protein-target assays, we confirm that models leveraging Cell Painting image-based profiles outperformed the baseline cell count model. We propose several other practical recommendations for benchmarking machine learning models for predicting bioactivity and assessing the added value of mRNA, protein, or image-based profiles.