View a PDF of the paper titled Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models, by Thomas Tulinski and 3 other authors
Abstract:Energy-based models (EBMs) are flexible generative architectures inspired by statistical physics, but their learning and generative properties remain poorly understood. Here, we analyze a solvable EBM in the high-dimensional limit: the spherical Boltzmann machine (SBM). Combining tools from random matrix theory and dynamical mean-field theory, we: derive and solve closed equations describing the training dynamics of the SBM in the high-dimensional limit; compute the asymptotic Bayesian evidence, which acts as a partition function in parameter space and encodes global properties of the trained model; and uncover cascades of phase transitions that occur both during training and as a function of hyperparameters, related to successive alignment and condensation of the top modes of the coupling matrix to the data. We connect these transitions to sampling-time generative phenomena in a teacher-student scenario, including: sampling temperature tuning, double descent as a function of regularization strength, tempered posterior effects, and out-of-equilibrium effects during training that induce biases in the trained model. We provide numerical evidence showing that all these phenomena appear in standard generative architectures, beyond the SBM.
Submission history
From: Jorge Fernandez-De-Cossio-Diaz [view email]
[v1]
Sat, 9 May 2026 16:15:18 UTC (1,101 KB)
[v2]
Wed, 18 Nov 2026 20:02:44 UTC (1,579 KB)
