Swapping to Remote Memory over InfiniBand: An Approach using a High Performance Network Block Device
Shuang Liang, Ranjit Noronha and Dhabaleswar K. Panda
IEEE International Conference on Cluster Computing (Cluster 2005)
Boston, Massachusetts, USA, September 27 - 30, 2005
Abstract
Traditionally, remote memory accesses in cluster systems are very expensive operations, which perform 20-100 times slower than local memory accesses. Modern RDMA capable networks such as InfiniBand and Quadrics provide low latency of a few microseconds and high bandwidth of up to 10 Gbps. This has made remote memory much closer to the local memory system. Using remote idle memory to enhance local memory hierarchy thus becomes an attractive choice, especially for data intensive applications in cluster environment. In this paper, we take the challenge to design a remote paging system for remote memory utilization in InfiniBand clusters. We present the design and implementation of a high performance networking blocking device (HPBD), which serves as a swap device for kernel Virtual Memory (VM) system for efficient page transfer to/from memory servers. Our experiments show that using HPBD, quick sort perform only 1.7 times slower than local memory system, and up to 21 times faster than local disk. And Our design is completely transparent to user applications. To the best of our knowledge, it is the first work of a remote pager design using InfiniBand for remote memory utilization.