Evolutionary adaptation proceeds through a small number of phenotypic modules.

Understanding how the myriad molecular impacts of mutation percolate to influence higher-order traits and ultimately fitness requires compressing a many-to-many mapping into something tractable. Decades of theoretical work suggest this may be possible because biological systems are modular: effects of perturbation are often funneled through particular pathways or subsystems rather than propagating freely through the organism. Yet, few empirical systems have been able to demonstrate such modularity at scale. Here, we show that, even across 774 diverse yeast lineages, fitness variation across 12 drug environments is organized by a strikingly low-dimensional structure defined by only a few inferred phenotypic axes that capture the main patterns of variation. Lineages drawn from multiple evolutionary histories reveal more of these phenotypic axes than those derived from a single selection pressure. Consistent with many of these lineages having evolved under strong selection pressure, their mutations often exhibit broad pleiotropy, affecting nearly all inferred phenotypic axes. However, fitness in any given drug depends on a much sparser subset of the phenotypic modules these axes reflect. By compressing many-to-many relationships, this low-dimensional framework exposes the modular phenotypic space, as well as the context-dependent contribution of each phenotypic module to fitness that can constrain the pleiotropic effects of adaptive mutations. It also highlights that the apparent complexity of genotype-phenotype-fitness maps depends not only on environmental context but also on the diversity of mutations through which they are observed, laying the groundwork for identifying the key phenotypic modules that matter for fitness.