Noether’s Theorem and Machine Learning
Emmy Noether discovered her eponymous theorem in the context of Einstein’s General Theory of Relativity. A couple of decades after her death, it became the foundation for modern particle physics. In the final chapter of my book about Noether’s Theorem I survey some of its recent applications far afield from either of these physics contexts—including applications outside of the physical sciences.
Since finishing the book I’ve become aware of some interesting applications that didn’t make it into the chapter. One of these is machine learning. I’ll briefly mention a few recent papers that might serve as a starting point for those interested in finding out more.
Several papers presented at the 35th Conference on Neural Information Processing Systems in 2021 explored extensions of Noether’s Theorem or related results that can lead to insights in machine learning problems.
In “Noether Networks: Meta-Learning Useful Conserved Quantities” Ferran Alet and coauthors “focus on sequential prediction problems and take inspiration from Noether’s theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities.”
In “Noether’s Learning Dynamics: Role of Symmetry Breaking in Neural Networks”, Hidenori Tanaka and Daniel Kunin exploit a close analogy with the Lagrangian formulation of classical mechanics, combined with the concept of symmetry breaking from particle physics, to establish a “theoretical foundation to discover geometric design principles for the learning dynamics of neural networks.”
Daniel Kunin and collaborators, in “Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics”, stay close to the concepts in Noether’s original theorem, finding differentiable symmetries in the training loss and relating these to conserved quantities. They believe that “symmetry has been underutilized in understanding the mechanics of neural networks.”
Christopher Gadzinski has put together a kind of tutorial titled “Where Is Noether’s Principle in Machine Learning?” that has some lovely interactive diagrams and contains a proof of a discrete Noether’s principle.
In the past few decades Noether’s Theorem has found application and provided inspiration in a rapidly increasing variety of disciplines and research programs. In Einstein’s Tutor I delve into why and how an abstruse mathematical result, derived by Emmy Noether to describe conservation laws in gravitation, has managed to spread tendrils of influence throughout and even beyond science.
