Developing an OpenSHMEM Model Over a Switchless PCIe Non-Transparent Bridge Interface

Citations Over Time

Abstract

OpenSHMEM is an emerging parallel programming model in distributed systems that supports partitioned global address space (PGAS). It was originally developed for Cray systems, and it was subsequently adopted by numerous vendors of the InfiniBand and Ethernet network interfaces, since most high-performance computing (HPC) systems deploy switch-based interconnect networks using InfiniBand and Ethernet technologies. Recently, Peripheral Component Interconnect Express (PCIe) has become one of the most promising candidates for deploying cost-effective popular HPC systems because of its low cost and powerful features, as well as the Non-Transparent Bridge(NTB) technology (interconnect interface for PCIe). However, there is little work done on implementing the OpenSHMEM library for PCIe systems. Herein, we introduce a prototype of a switchless interconnect network with PCIe NTB. In our switchless interconnect network system, the computing nodes are interconnected via the PCIe NTB interface. Based on the PCIe NTB interconnect network, we implemented the OpenSHMEM programming interface to support the PGAS mechanism for PCIe NTB interconnect networks. Our design and implementation of an OpenSHMEM programming model via PCIe NTB shows that is feasible and possible to have a PCIe-based, cost-effective, high performance interconnect network in a high-performance programming model.

Related Papers

• → Performance Analysis and Evaluation of InfiniBand FDR and 40GigE RoCE on HPC and Cloud Computing Systems(2012)50 cited
• → Evaluating InfiniBand performance with PCI Express(2005)28 cited
• → Compatibility enhancement and performance measurement for socket interface with PCIe interconnections(2019)10 cited
• → Improving Strong-Scaling on GPU Cluster Based on Tightly Coupled Accelerators Architecture(2015)
• → Design and Implementation of JNI Interface of PCIe NTB Interconnect Network for RDMA-based HDFS(2021)