Parts | Days | Selection | Search | Updates | Downloads | Help

T: Fachverband Teilchenphysik

T 44: Data, AI, Computing 4 (workflow)

T 44.5: Talk

Tuesday, March 5, 2024, 17:00–17:15, Geb. 30.34: LTI

Orchestrated columnar-based analysis with Columnflow — •Mathis Frahm, Johannes Haller, Philip Keicher, Nathan Provoust, Marcel Rieger, Daniel Savoiu, Peter Schleper, Matthias Schröder, and Bogdan Wiederspan — Institut für Experimentalphysik, Universität Hamburg

The large datasets and increasing complexity of modern physics analysis in high energy collider physics pose a major challenge to the analysis workflows. Systems are required that can process large scales of data efficiently, while keeping the execution of the full analysis manageable. In this talk, we present Columnflow, a tool for columnar-based data analysis. Columnflow provides an orchestrated, yet flexible workflow that handles the bookkeeping of results and dependencies automatically. Typical analysis tasks such as propagating systematic uncertainties, machine-learning applications, and statistical inference are transparently included in the workflow. The implemented workflow allows the usage of distributed computing resources and is fully configurable but at the same time accessible to newcomers.

Keywords: Framework; Workflow; Columnar; Orchestration; Law