Remote Direct Memory Access (RDMA) as a communication paradigm is built around the concept of user-level networking. Three core design principles — namely complete separation of data transfer and communication resource control operations, asynchronous data transfer and completion, and explicit management of communication buffer ownership — enable very efficient, zero data copy application-to-application data transfers while bypassing the operating system at both source and sink of an RDMA communication operation. For complete and safe operating system bypass during data transfer operations, RDMA relies on offloading the communication stack to the network adapter.
By contrasting typical RDMA communication stack implementations, SoftiWARP intentionally avoids any hardware dependency demanded by stateful protocol offloading and implements all protocol processing in software. Comprising a Linux kernel module and a user space communication library, SoftiWARP seamlessly integrates with the industry standard OpenFabrics RDMA host infrastructure.
SoftiWARP appears as a regular OpenFabrics RDMA device to the application, while implementing the IETF iWARP RDMA protocol suite on top of the in-kernel TCP stack. From an application point of view, it turns any local Ethernet device into an RDMA-capable interface (RNIC).
SoftiWARP provides the application with the benefits of a semantically rich RDMA communication interface, such as one-sided remote data read and write operations, and selectable operation completion policies.
The implementation is able to turn those explicit RDMA communication semantics into efficient protocol processing, which helps it outperform a typical send/receive TCP socket application setup on specific communication patterns.
For example, a remote RDMA read operation can be served at the peer side by in-kernel SoftiWARP without involving any application scheduling. Furthermore, at the sending side, zero copy data transfer can be achieved.
SoftiWARP is wire-compatible with any other (hardware) RDMA device. Hence, it enables mixed setups with some systems equipped with RDMA hardware and other systems running the software RDMA-only stack. This is particularly interesting for client-server setups where only the server is equipped with more expensive RDMA hardware.
SoftiWARP is an open-source project of the IBM Research – Zurich Laboratory. It is available online under selectable BSD/MIT or GPLv2 license terms.