Dynamic nested hierarchies: self-evolving machine learning architectures for lifelong learning.

Contemporary machine learning models, including large language models, exhibit remarkable capabilities in static tasks yet falter in non-stationary environments due to rigid architectures that hinder continual adaptation and lifelong learning. Building upon the nested learning (NL) paradigm, which decomposes models into multi-level optimization problems with fixed update frequencies, this work proposes Dynamic Nested Hierarchies (DNH) as an extension enabling autonomous structural adaptation. Unlike static nested learning where hierarchy depth and update frequencies are fixed at initialization, DNH introduces three biologically-grounded mechanisms: (1) level addition triggered by meta-loss thresholds, analogous to adult neurogenesis in the hippocampal dentate gyrus; (2) level pruning based on gradient contribution, analogous to synaptic elimination; and (3) frequency modulation driven by local surprise signals, analogous to neural oscillation adaptation. We provide explicit mappings between these mechanisms and neuroplasticity processes, moving beyond superficial analogy to principled design. Through rigorous mathematical formulations, we prove convergence bounds of