Freiburg 2019 – scientific programme
Sessions | Days | Selection | Search | Updates | Downloads | Help
FM: Fall Meeting
FM 55: Quantum & Information Science: Neural Networks, Machine Learning, and Artificial Intelligence II
FM 55.3: Talk
Wednesday, September 25, 2019, 14:45–15:00, 1098
Metric Gaussian Variational Inference — •Jakob Knollmüller1,2 and Torsten Enßlin1,2 — 1Max-Planck Institute for Astrophysics, Garching — 2Ludwig-Maximilian University, Munich
A variational Gaussian approximation of the posterior distribution can be an excellent way to infer posterior quantities. However, to *capture all posterior correlations the parametrization of the full covariance is required, which scales quadratic with the problem size. This scaling prohibits full-covariance approximations for large-scale problems. As a solution to this limitation we propose Metric Gaussian Variational Inference (MGVI). This procedure approximates the variational covariance such that it requires no parameters on its own and still provides reliable posterior correlations and uncertainties for all model parameters. We approximate the variational covariance with the inverse Fisher metric, a local estimate of the true posterior uncertainty. This covariance is only stored implicitly and all necessary quantities can be extracted from it by independent samples drawn from the approximating Gaussian. MGVI requires the minimization of a stochastic estimate of the Kullback-Leibler divergence only with respect to the mean of the variational Gaussian, a quantity that scales linearly with the problem size. We motivate the choice of this covariance from an information geometric perspective. We validate the method against established approaches, demonstrate its scalability into the regime over a million parameters and capability to capture posterior distributions over complex models with multiple components and strongly non-Gaussian prior distributions. (J. Knollmüller et al. (2019) ArXiv:1901.11033)