Berlin 2024 – wissenschaftliches Programm
Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe
DY: Fachverband Dynamik und Statistische Physik
DY 19: Machine Learning in Dynamics and Statistical Physics II (joint session DY/SOE)
DY 19.8: Vortrag
Dienstag, 19. März 2024, 11:30–11:45, BH-N 243
Collective Variables for Neural Networks — •Konstantin Nikolaou1, Samuel Tovey1, Sven Krippendorf2, and Christian Holm1 — 1Institute for Computational Physics, University of Stuttgart, Germany — 2Arnold Sommerfeld Center for Theoretical Physics, Ludwig-Maximilians-Universität, Germany
Neural Networks have witnessed extensive integration across diverse domains within physics. However, our focus shifts towards the inverse problem: How can neural networks benefit from physics? Learning with a neural network involves algorithmically assimilating information into a model. Nevertheless, the process of neural learning remains largely elusive, given the challenge of understanding how to extract information from its dynamics. Analogous to dynamical systems in statistical physics, describing neural network training involves an extensive number of degrees of freedom, which appears to benefit from a description through macroscopic quantities. To that end, we introduce Collective Variables for neural networks, tracing out microscopic degrees of freedom to describe and analyze the learning process at every stage. We investigate the initial state as well as the learning dynamics of the network in the context of the Collective Variables. We find a correlation between the initial network state and the generalization of the model computed after training. Moreover, we use the collective variables to identify and analyze stages arising in the dynamics of the learning process.
Keywords: collective variables; neural networks; learning theory; machine learning; dynamical systems