Exploring Data Staging Across Deep Memory Hierarchies for Coupled Data Intensive Simulation Workflows
Citations Over TimeTop 10% of 2015 papers
Abstract
As applications target extreme scales, data staging and in-situ/in-transit data processing have been proposed to address the data challenges and improve scientific discovery. However, further research is necessary in order to understand how growing data sizes from data intensive simulations coupled with the limited DRAM capacity in High End Computing systems will impact the effectiveness of this approach. In this paper, we explore how we can use deep memory levels for data staging, and develop a multi-tiered data staging method that spans bothDRAM and solid state disks (SSD). This approach allows us to support both code coupling and data management for data intensive simulation workflows. We also show how an adaptive application-aware data placement mechanism can dynamically manage and optimize data placement across the DRAM ands storage levels in this multi-tiered data staging method. We present an experimental evaluation of our approach using wolf resources: an Infiniband cluster (Sith) and a Cray XK7system (Titan), and using combustion (S3D) and fusion (XGC1) simulations.
Related Papers
- → InfiniBand Verbs Optimizations for Remote GPU Virtualization(2015)3 cited
- → Tuning remote GPU virtualization for InfiniBand networks(2016)6 cited
- → InfiniBand & OpenFabrics---InfiniBand and OpenFabrics at SC06(2006)1 cited
- Research of the Switching Based on InfiniBand(2004)
- → Modelling and simulation of IP over InfiniBand with OMNeT++(2017)