Berlin 2024 – scientific programme

Parts | Days | Selection | Search | Updates | Downloads | Help

SOE: Fachverband Physik sozio-ökonomischer Systeme

SOE 2: Focus Session: Machine Learning for Complex Socio-economic Systems

SOE 2.5: Talk

Monday, March 18, 2024, 11:00–11:15, MA 001

Towards a complex systems theory of attention? — •Claudius Gros — Institute for Theoretical Physics, Goethe University Frankfurt

The attention mechanism is at the core of the current AI hype. It powers transformers and hence all modern large language models, such as GPT or LLaMA. Classical deep learning model are optimized for information processing, whereas attention allows for information routing. It is argued, that the modeling techniques used complex systems theory and physics can contribute to an understanding of what is going on inside transformers. On this background an introduction to attention is presented.

Keywords: complex systems theory; attention; machine learning