Würzburg 2018 – scientific programme

Parts | Days | Selection | Search | Updates | Downloads | Help

T: Fachverband Teilchenphysik

T 16: GRID Computing / Experimentelle Methoden I

T 16.1: Talk

Monday, March 19, 2018, 16:00–16:15, Z6 - SR 2.006

Dynamic Integration and Scheduling of Opportunistic Resources — •Matthias Jochen Schnepf, Christoph Heidecker, Manuel Giffels, and Günter Quast — Karlsruhe Institute of Technology

The demand for computing resources in high energy physics (HEP) varies over time due to conferences and periods of data taking. To cover such peak loads the integration of opportunistic resources on-demand can be used. However, HEP software needs a specific software environment which is usually not provided by opportunistic resources. This makes it necessary to use container or virtualization technologies which provide the HEP software environment on opportunistic resources.

The CMS group at the Karlsruhe Institute of Technology developed the ROCED cloud scheduler to dynamically provision, integrate and manage opportunistic resources in combination with the HTCondor batch system and modern virtualization and container technologies. The transparent integration of opportunistic resources into a single batch system allows users to access thousands of additional CPU cores without resource specific customizations. However, it turned out that network limitations in conjunction with I/O intensive tasks can cause CPU inefficiencies on opportunistic sites. In order to reduce these inefficiencies, we currently working on a resource scheduling based on the available network bandwidth and the I/O demands of individual jobs. In this presentation, an overview of developed technologies, integrated resources and the status of I/O based job scheduling will be given.