HEP Applications Experience with the European DataGrid Middleware and Testbed

Abstract

The European DataGrid (EDG) project ran from 2001 to 2004, with the aim of producing middleware which could form the basis of a production Grid, and of running a testbed to demonstrate the middleware. HEP experiments (initially the four LHC experiments and subsequently BaBar and D0) were involved from the start in specifying requirements, and subsequently in evaluating the performance of the middleware, both with generic tests and through increasingly complex data challenges. A lot of experience has therefore been gained which may be valuable to future Grid projects, in particular LCG and EGEE which are using a substantial amount of the middleware developed in EDG. We report our experiences with job submission, data management and mass storage, information and monitoring systems, Virtual Organisation management and Grid operations, and compare them with some typical Use Cases defined in the context of LCG. We also describe some of the main lessons learnt from the project, in particular in relation to configuration, fault-tolerance, interoperability and scalability, as well as the software development process itself, and point out some areas where further work is needed. We also make some comments on how these issues are being addressed in LCG and EGEE.

Related Papers

• → Experiences with the gLite Grid Middleware(2005)6 cited
• → LCG Conditions Database Project Overview(2005)6 cited
• → Experience integrating a General Information System API in LCG Job Management and Monitoring Services(2005)3 cited
• → Software management infrastructure in the LCG Application Area(2005)
• The EGEE user support infrastructure(2007)