RDMA-capable interconnects, which provide ultra-low latency and high-bandwidth, are increasingly being used in the context of distributed storage and data processing systems. However, the deployment of such systems in virtualized data centers is currently inhibited by the lack of a flexible and high-performance virtualization solution for RDMA network interfaces. Furthermore, state-of-the-art hardware virtualization is complex and takes up a considerable amount of die space.
Traditional network interface cards (NICs) do not have a sufficient form of device isolation to allow direct sharing between virtual machines and thus, network I/O requires supervision by the hypervisor when using software virtualization. However, going through the hypervisor on every I/O operation is expensive.
In contrast, RDMA devides already offer isolation at the level of user-application, which can be directly leveraged for virtualization. To achieve that, RDMA builds on the concept of separation of paths for control and data operations.
HyV (hybrid virtualization) extends this concept from user applications to virtual machines. With this hybrid virtualization architecture, RDMA control operations are virtualized using hypervisor involvement, whereas data operations are set up to bypass the hypervisor completely without the need for hardware virtualization.
HyV does not require rewriting applications for virtualized RDMA as it integrates into the widely used OpenFabrics software stack.
References
Jonas Pfefferle, Patrick Stuedi, Animesh Trivedi, Bernard Metzler, Ionnis Koltsidas, and Thomas R. Gross
"A Hybrid I/O Virtualization Framework for RDMA-capable Network Interfaces"
In Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '15) 2015.
Design overview