NetKV: Scalable, self-managing, load balancing as a network function

Abstract

Distributed key-value systems (e.g., memcached) are critical tools to cache popular content in memory, which avoids complex and expensive database queries and file system accesses. To efficiently use cache resources, balancing the load across a cluster of cache servers is important. Current approaches place a proxy at each client that can redirect requests across the cluster, but this requires modification to each client and makes dynamic replication of keys difficult. While a centralized proxy can be used, this traditionally has not been scalable. We design and implement NetKV, a scalable, self-managing, load balancer for memcached clusters. NetKV exploits recent advances in Network Function Virtualization to provide efficient packet processing in software, producing a high performance, centralized proxy that can forward over 10.5 million requests per second. NetKV efficiently and accurately detects hot keys using stream-analytic techniques, then replicates them to meet the allowed load imbalance bound set by administrators. NetKV uses "balls and bins" load analysis to adaptively determine the replication factor and set of hot keys. Our prototype adds minimal latency to each request, and our algorithms effectively balance load in both a 12 server cluster and a large-scale simulation driven by a trace of wikipedia requests.