RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store

Disk-based storage is becoming increasingly problematic in meeting the needs of large-scale cloud applications. Recently RAM-based storage is proposed by aggregating the RAM of thousands of commodity servers in data center networks (DCN). These studies focus on improving performance with low latency RPC and fast failure recovery. RAM-based storage brings great DCN-related challenges, e.g., false server failure detection due to network problems, traffic congestion during failure recovery, and ToR switch failure handling. This paper presents RAMCube, a DCN-oriented design for RAM-based key-value store based on the BCube network [9]. RAMCube exploits the properties of BCube to restrict all failure detection and recovery traffic within one-hop neighborhood, and leverages BCube's multiple paths to handle switch failures. Prototype implementation demonstrates that RAMCube is promising to achieve high performance I/O and fast failure recovery in large-scale DCNs.

[1]  GhemawatSanjay,et al.  The Google file system , 2003 .

[2]  Wilson C. Hsieh,et al.  Bigtable: A Distributed Storage System for Structured Data , 2006, TOCS.

[3]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[4]  Haitao Wu,et al.  ServerSwitch: A Programmable and High Performance Platform for Data Center Networks , 2011, NSDI.

[5]  Michael Stonebraker,et al.  H-store: a high-performance, distributed main memory transaction processing system , 2008, Proc. VLDB Endow..

[6]  Dharma P. Agrawal,et al.  Generalized Hypercube and Hyperbus Structures for a Computer Network , 1984, IEEE Transactions on Computers.

[7]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[8]  Parag Agrawal,et al.  The case for RAMClouds: scalable high-performance storage entirely in DRAM , 2010, OPSR.

[9]  Haitao Wu,et al.  MDCube: a high performance network structure for modular data center interconnection , 2009, CoNEXT '09.

[10]  Michael I. Jordan,et al.  Managing data transfers in computer clusters with orchestra , 2011, SIGCOMM.

[11]  Hector Garcia-Molina,et al.  Main Memory Database Systems: An Overview , 1992, IEEE Trans. Knowl. Data Eng..

[12]  Jin Li,et al.  SkimpyStash: RAM space skimpy key-value store on flash-based storage , 2011, SIGMOD '11.

[13]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[14]  Mendel Rosenblum,et al.  It's Time for Low Latency , 2011, HotOS.

[15]  Nong Xiao,et al.  PIBUS: A Network Memory-Based Peer-to-Peer IO Buffering Service , 2007, Networking.

[16]  Mendel Rosenblum,et al.  Fast crash recovery in RAMCloud , 2011, SOSP.

[17]  Fred B. Schneider,et al.  The primary-backup approach , 1993 .

[18]  Abhinav Dutta.  Cheap and Large CAMs for High Performance Data-Intensive Networked Systems , .

[19]  Dejan S. Milojicic,et al.  Distributed Line Graphs: A Universal Technique for Designing DHTs Based on Arbitrary Regular Graphs , 2012 .

[20]  Amar Phanishayee,et al.  FAWN: a fast array of wimpy nodes , 2009, SOSP '09.

[21]  Amin Vahdat,et al.  Hedera: Dynamic Flow Scheduling for Data Center Networks , 2010, NSDI.

[22]  Dean Daniels,et al.  Oracle's symmetric replication technology and implications for application design , 1994, SIGMOD '94.

[23]  Mahadev Konar,et al.  ZooKeeper: Wait-free Coordination for Internet-scale Systems , 2010, USENIX ATC.

[24]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[25]  Armando Fox,et al.  Toward Recovery-Oriented Computing , 2002, VLDB.

[26]  Jin Li,et al.  FlashStore , 2010, Proc. VLDB Endow..