Towards scalable distributed graph database engine for hybrid clouds

Abstract

Large graph data management and mining in clouds has become an important issue in recent times. We propose Acacia which is a distributed graph database engine for scalable handling of such large graph data. Acacia operates between the boundaries of private and public clouds. Acacia partitions and stores the graph data in the private cloud during its initial deployment. Acacia bursts into the public cloud when the resources of the private cloud are insufficient to maintain its service-level agreements. We implement Acacia using X10 programming language. We describe how Top-K PageRank has been implemented in Acacia. We report preliminary experiment results conducted with Acacia on a small compute cluster. Acacia is able to upload 69 million edges LiveJournal social network data set in about 10 minutes. Furthermore, Acacia calculates the average out degree of vertices of LiveJournal graph in 2 minutes. These results indicate Acacias potential for handling large graphs.