The LHCb DIRAC-based production and data management operations systems
Citations Over TimeTop 12% of 2012 papers
Abstract
The LHCb computing model was designed in order to support the LHCb physics program, taking into account LHCb specificities (event sizes, processing times etc...). Within this model several key activities are defined, the most important of which are real data processing (reconstruction, stripping and streaming, group and user analysis), Monte-Carlo simulation and data replication. In this contribution we detail how these activities are managed by the LHCbDIRAC Data Transformation System. The LHCbDIRAC Data Transformation System leverages the workload and data management capabilities provided by DIRAC, a generic community grid solution, to support data-driven workflows (or DAGs). The ability to combine workload and data tasks within a single DAG allows to create highly sophisticated workflows with the individual steps linked by the availability of data. This approach also provides the advantage of a single point at which all activities can be monitored and controlled. While several interfaces are currently supported (including python API and CLI), we will present the ability to create LHCb workflows through a secure web interface, control their state in addition to creating and submitting jobs. To highlight the versatility of the system we present in more detail experience with real data of the 2010 and 2011 LHC run.
Related Papers
- → PyMOOSE: Interoperable scripting in Python for MOOSE(2008)109 cited
- USING PYTHON AND EPPY FOR A LARGE NATIONAL SIMULATION STUDY(2016)
- → Data analysis with MATLAB® or Python(2022)4 cited
- → User-friendly parallelization of GAUDI applications with Python(2010)1 cited
- → Scripting(2011)