Integrating recent non-volatile memory (NVM) technology such as Flash, MRAM or PCM with the traditional block device I/O stack unnecessarily restricts the resultant system performance improvements and hides novel, media specific advantages of the new technologies. The block device interface as the traditional device abstraction layer was introduced for bridging the gap between fast, random DRAM access and the high-latency, serial access to electro-mechanical storage devices. In contrast, the media physics of NVM enables much smaller access latencies and the overall performance even benefits from concurrent, parallel media access.
To overcome those limitations of the current I/O stack, we are working towards tight integration of storage I/O with today’s high-performance RDMA host network stack. Core semantics of RDMA, such as application private deep I/O queues and asynchronous remote read and write operations, as well as the possibility to implement save zero copy data movement between storage device and application buffer are essential features to serve uncompromised storage performance to the application.
To instantiate that architecture, we implemented a device driver for local NVM access called DSA (Direct Storage Access). It uses the OpenFabrics RDMA host infrastructure to provide an application level, asynchronous interface to open, read and write attached NVM. As any OpenFabrics compliant RDMA device, it comprises a kernel module and a user library. DSA today offers RDMA semantics to access flash memory (explicit communication buffer management, RDMA Read/Write operations, asynchronous work completions, etc.).
To abstract from the specifics of any flash device, DSA further introduces a hardware abstraction layer (SAL - Storage Abstraction Layer) which interfaces with hardware device specific functions. An optional kernel-level NVP client module enables a block device on NVM for high-performance legacy access through the local file system. While we are constantly improving the driver, we plan to open source the DSA code soon.
