A Tale of Free Energies. With Max Welling (University of Amsterdam, CuspAI) and Lars Holdijk (University of Oxford). Cambridge University Press, July 2026.
Free energy is all you need 🔗
Probabilistic machine learning and nonequilibrium thermodynamics are, to a large extent, the same mathematics. The book develops a single lens, the variational free energy, that unifies variational autoencoders, normalizing flows, diffusion models, optimal control, and generative flow networks, and ties their training objectives to entropy production and the fluctuation theorems.
Variational autoencoders stand at the crossroads of physics’ variational principles and machine learning’s variational inference, sharing a common thread through the concept of free energy.
One dictionary, two fields 🔗
A recurring map runs through the book. Latent variables play the role of physical degrees of freedom, the partition function becomes the marginal likelihood, the free energy becomes the negative log-likelihood, and the physicist’s variational free-energy bound is exactly the ELBO that trains a VAE.
What it covers 🔗
About 300 pages across 24 chapters in three parts: mathematical preliminaries, discrete-time processes and their thermodynamics, and the continuous-time domain. Along the way it connects entropy production to the rate of change of a KL divergence, the Jarzynski and Crooks fluctuation theorems to partition-function and likelihood estimation, Schrödinger bridges and optimal transport to finite-time generative sampling, and score-based diffusion to Langevin dynamics. It grew out of lectures at the African Institute for Mathematical Sciences.
Endorsements 🔗
Just as thermodynamics proved key to understanding the age of steam, stochastic thermodynamics will prove key to understanding the age of AI. This book is the first comprehensive guide to the principles of stochastic thermodynamics and how they relate to modern AI. It is much needed and will be widely read.
Neil Lawrence, University of Cambridge
Generative AI now shapes science and industry, but its conceptual underpinnings are often opaque even to those who use it daily. This text develops an elegant unifying perspective grounded in the physics of stochastic thermodynamics — an angle no other book has explored at this depth. An inspiring resource for researchers in both fields.
Miranda Cheng, Academia Sinica
In Generative AI and Stochastic Thermodynamics, Max Welling achieves something rare and thrilling: a beautiful marriage of two deep and historically separate fields, weaving together the principles of modern AI with the elegant formalism of statistical physics. Complex ideas are presented with remarkable clarity and care, never sacrificing mathematical rigor for the sake of accessibility, yet remaining wonderfully approachable throughout. This book is an essential read for anyone working at the intersection of AI and the physical sciences, and I have no doubt it will inspire a new generation of cross-disciplinary thinking.
Rose Yu, UC San Diego
Generative AI and statistical physics keep rediscovering each other’s ideas under different names. This book presents both fields in the same framework and is the most interesting textbook I have read this year. It taught me new things about areas I thought I knew well. I strongly recommend it for anyone interested in AI and physics.
Jascha Sohl-Dickstein, Anthropic
With further endorsement from Geoffrey Hinton (University of Toronto, 2024 Nobel Laureate in Physics and 2018 Turing Award).