Göttingen 2025 – wissenschaftliches Programm

Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe

AKPIK: Arbeitskreis Physik, moderne Informationstechnologie und Künstliche Intelligenz

AKPIK 1: Theory of Machine Learning (joint session MP/AKPIK)

AKPIK 1.2: Vortrag

Mittwoch, 2. April 2025, 14:05–14:25, ZHG001

Opening the Black Box: predicting the trainability of deep neural networks with reconstruction entropy — •Yanick Thurn¹, Ro Jefferson², and Johanna Erdmenger¹ — ¹Institute for Theoretical Physics and Astrophysics, Julius-Maximilians-University Wuerzburg — ²Institute for Theoretical Physics, and Department of Information and Computing Sciences, Utrecht University

An important challenge in machine learning is to predict the initial conditions under which a given neural network will be trainable. We present a method for predicting the trainable regime in parameter space for deep feedforward neural networks (DNNs) based on reconstructing the input from subsequent activation layers via a cascade of single-layer auxiliary networks. We show that a single epoch of training of the shallow cascade networks is sufficient to predict the trainability of the deep feedforward network on a range of datasets (MNIST, CIFAR10, FashionMNIST, and white noise). Moreover, our approach illustrates the networks decision making process by displaying the changes performed on the input data at each layer, which we demonstrate for both a DNN trained on MNIST and the vgg16 CNN trained on the ImageNet dataset.

Keywords: artificial intelligence; neural networks; xai