Berlin 2024 – wissenschaftliches Programm

Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe

DY: Fachverband Dynamik und Statistische Physik

DY 19: Machine Learning in Dynamics and Statistical Physics II (joint session DY/SOE)

DY 19.12: Vortrag

Dienstag, 19. März 2024, 12:30–12:45, BH-N 243

Investigating the Evolution of Fisher Information for Neural Network Dynamics — •Marc Sauter, Samuel Tovey, Konstantin Nikolaou, and Christian Holm — Institute for computational physics, Stuttgart, Germany

Machine learning has proven to be a powerful tool with remarkable effectiveness for various physical applications. Even though Neural Networks offer great potential for solving complex tasks, their black-box nature limits the information obtainable about the underlying mechanisms they employ. Especially in physics, where the exact methodologies of solving tasks are just as important as finding solutions itself, advances in interpretable machine learning promise to aid the applicability of Neural Networks task-solving capabilities greatly. The Fisher Information Matrix (FIM) is a statistical measure known for its ability to identify second order phase transitions in physical systems. It can also be used for analyzing learning dynamics, where it expresses correlations between influences of parameters in Neural Networks. However, because of its size, computation of the FIM is currently intractable for Neural Networks used in common problem settings. As a way of obtaining parts of the information contained in the FIM, we introduce a novel mathematical relationship between the trace of the FIM and the Neural Tangent Kernel, a smaller observable of neural network training. We apply this approach for simple test models and discuss arising research topics.

Keywords: Machine Learning; Fisher Information; Neural Tangent Kernel; Phase Transitions