Berlin 2024 – scientific programme

Parts | Days | Selection | Search | Updates | Downloads | Help

DY: Fachverband Dynamik und Statistische Physik

DY 19: Machine Learning in Dynamics and Statistical Physics II (joint session DY/SOE)

DY 19.13: Talk

Tuesday, March 19, 2024, 12:45–13:00, BH-N 243

Near-zero-cost post-training uncertainties for deep learning architectures — •Filippo Bigi, Sanggyu Chong, Michele Ceriotti, and Federico Grasselli — Laboratory of Computational Science and Modeling (COSMO), IMX, École Polytechnique Fédérale de Lausanne, Switzerland

Over the last decade, deep learning models have shown impressive performance and versatility on an extremely wide range of tasks. However, their probability estimates are unreliable, especially outside of the training distribution, with neural networks often returning overconfident results when queried on unfamiliar data. Although several uncertainty quantification schemes are available, their practical downsides hinder their widespread adoption. We propose a novel method for estimating the predictive uncertainties of deep learning architectures based on the interpretation of the last layer of neural networks as a linear Gaussian process. Contrary to previous methods, the proposed approach is simple, scalable, does not involve modification of the architecture or the training procedure, can be applied to trained models a posteriori, and generates uncertainty estimates with a single forward pass at negligible additional cost. We demonstrate the accuracy and practicality of our scheme on a wide range of machine learning datasets.

Keywords: Neural networks; Uncertainty estimation; Gaussian process; Bayesian