Efficient data transfer protocols for big data

Citations Over TimeTop 10% of 2012 papers

Abstract

Data set sizes are growing exponentially, so it is important to use data movement protocols that are the most efficient available. Most data movement tools today rely on TCP over sockets, which limits flows to around 20Gbps on today's hardware. RDMA over Converged Ethernet (RoCE) is a promising new technology for high-performance network data movement with minimal CPU impact over circuit-based infrastructures. We compare the performance of TCP, UDP, UDT, and RoCE over high latency 10Gbps and 40Gbps network paths, and show that RoCE-based data transfers can fill a 40Gbps path using much less CPU than other protocols. We also show that the Linux zero-copy system calls can improve TCP performance considerably, especially on current Intel “Sandy Bridge”-based PCI Express 3.0 (Gen3) hosts.

Related Papers

• → PROP: Using PCIe-Based RDMA to Accelerate Rack-Scale Communications in Data Centers(2015)5 cited
• → PCIe Side-channel Attack on I/O Device via RDMA-enabled Network Card(2022)3 cited
• → A multi-port 10GbE PCIe NIC featuring UDP offload and GPUDirect capabilities.(2015)3 cited
• → High Performance RDMA-Based Daq Platform Over PCIe Routable Network(2018)2 cited
• → Design and Implementation of JNI Interface of PCIe NTB Interconnect Network for RDMA-based HDFS(2021)