Scaling Deep Learning Workloads: NVIDIA DGX-1/Pascal and Intel Knights Landing
Citations Over TimeTop 10% of 2017 papers
Abstract
Deep Learning (DL) algorithms have become ubiquitous in data analytics. As a result, major computing vendors - including NVIDIA, Intel, AMD and IBM - have architectural road-maps influenced by DL workloads. Furthermore, several vendors have recently advertised new computing products as accelerating DL workloads. Unfortunately, it is difficult for data scientists to quantify the potential of these different products. This paper provides a performance and power analysis of important DL workloads on two major parallel architectures: NVIDIA DGX-1 (eight Pascal P100 GPUs interconnected with NVLink) and Intel Knights Landing (KNL) CPUs interconnected with Intel Omni-Path. Our evaluation consists of a cross section of convolutional neural net workloads: CifarNet, CaffeNet, AlexNet and GoogleNet topologies using the Cifar10 and ImageNet datasets. The workloads are vendor optimized for each architecture. GPUs provide the highest overall raw performance. Our analysis indicates that although GPUs provide the highest overall performance, the gap can close for some convolutional networks; and KNL can be competitive when considering performance/watt. Furthermore, NVLink is critical to GPU scaling.
Related Papers
- → High Order Seismic Simulations on the Intel Xeon Phi Processor (Knights Landing)(2016)30 cited
- → Amber PME Molecular Dynamics Optimization(2015)1 cited
- → The Next-generation Supercomputer and Visuakization(2006)
- Multi-level Structure Abstract and Description of Supercomputer(2008)
- Big Blue in the Bottomless Pit: The Early Years of IBM Chile(2008)