Dresden 2020 – scientific programme

The DPG Spring Meeting in Dresden had to be cancelled! Read more ...

Parts | Days | Selection | Search | Updates | Downloads | Help

AKBP: Arbeitskreis Beschleunigerphysik

AKBP 14: Focus session: Machine Learning

AKBP 14.4: Invited Talk

Thursday, March 19, 2020, 11:00–11:30, MOL 213

Advances in reinforcement learn for accelerator tuning at CERN — •Simon Hirlaender — University of Malta / CERN

The preservation of reliable and stable performance of accelerator complexes demands arduous effort. Hence to fully exploit the potential of accelerator-based facilities around the world, tackling unavoidable problems as drifts, hysteresis or fast set-up after configuration changes in an automated, efficient manner has become increasingly important. Well-known numerical optimization algorithms, as well as modern techniques based on reinforcement learning (RL), found their way into the control rooms for that purpose. This talk will address the main challenges with RL for accelerator operation and discuss recent progress at the CERN accelerator complex, especially at the LINAC4 accelerator and the AWAKE electron line. Various successful tests employing model-free optimization as well as highly sample efficient deep RL algorithms with more than ten degrees of freedom for problems such as trajectory steering and optics matching are covered. The method of transfer learning, where the controller was trained purely on simulated data, could be demonstrated. Besides, the training of a controller in disentangled latent space representations of image-based measurements was shown. The next boost in sample efficiency is expected from model-based reinforcement learning algorithms that learn the dynamics of a particular process explicitly and in this way, can reduce the interaction with the real environment as the accelerator time by orders of magnitude. In many cases, RL training is unfeasible otherwise. These models are uncertainty aware and offer promising properties for a wide range of possible applications, where an apriori model is not available. These studies mark the first mile-stones towards a self-tuning accelerator at CERN.