Space Bounds for Reliable Storage: Fundamental Limits of Coding

We study the inherent space requirements of reliable storage algorithms in asynchronous distributed systems. A number of recent works have used codes in order to achieve a better storage cost than the well-known replication approach. However, a closer look reveals that they incur extra costs in certain scenarios. Specifically, if multiple clients access the storage concurrently, then existing asynchronous code-based algorithms may store a number of copies of the data that grows linearly with the number of concurrent clients. We prove here that this is inherent. Given three parameters, (1) the data size -- D bits, (2) the concurrency level -- c, and (3) the number of storage node failures that need to be tolerated -- f, we show a lower bound of Omega(min(f,c)D) bits on the space complexity of asynchronous distributed storage algorithms. Intuitively, this implies that the asymptotic storage cost is either as high as with replication, namely O(fD), or as high under concurrency as with the aforementioned code-based algorithms, i.e., O(cD). We further present a technique for combining erasure codes with replication so as to obtain the best of both. We present an adaptive f-tolerant storage algorithm whose storage cost is O(min(f,c)D). Together, our results show that the space complexity of providing reliable storage in asynchronous distributed systems is Theta(min(f,c)D).

[1]  Idit Keidar,et al.  Byzantine disk paxos: optimal resilience with byzantine shared memory , 2004, PODC '04.

[2]  Dan Dobre,et al.  Space Bounds for Reliable Multi-Writer Data Store: Inherent Cost of Read/Write Primitives , 2015, ArXiv.

[3]  Leslie Lamport,et al.  On interprocess communication , 1986, Distributed Computing.

[4]  Rachid Guerraoui,et al.  Optimistic Erasure-Coded Distributed Storage , 2008, DISC.

[5]  Nancy A. Lynch,et al.  Information-Theoretic Lower Bounds on the Storage Cost of Shared Memory Emulation , 2016, PODC.

[6]  Jennifer L. Welch,et al.  Multiwriter Consistency Conditions for Shared Memory Registers , 2011, SIAM J. Comput..

[7]  Stefano Tessaro,et al.  Optimal Resilience for Erasure-Coded Byzantine Distributed Storage , 2005, International Conference on Dependable Systems and Networks (DSN'06).

[8]  Yehuda Afek,et al.  Benign Failure Models for Shared Memory (Preliminary Version) , 1993, WDAG.

[9]  Leslie Lamport,et al.  Interprocess Communication , 2020, Practical System Programming with C.

[10]  Michael K. Reiter,et al.  Low-overhead byzantine fault-tolerant storage , 2007, SOSP.

[11]  Rachid Guerraoui,et al.  Amnesic Distributed Storage , 2007, DISC.

[12]  Emmanuelle Anceaume,et al.  DataCube: A P2P Persistent Data Storage Architecture Based on Hybrid Redundancy Schema , 2010, 2010 18th Euromicro Conference on Parallel, Distributed and Network-based Processing.

[13]  Hagit Attiya,et al.  Sharing memory robustly in message-passing systems , 1990, PODC '90.

[14]  Nancy A. Lynch,et al.  A coded shared atomic memory algorithm for message passing architectures , 2014, 2014 IEEE 13th International Symposium on Network Computing and Applications.

[15]  Marcos K. Aguilera,et al.  Using erasure codes efficiently for storage in a distributed system , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[16]  Sam Toueg,et al.  Fault-tolerant wait-free shared objects , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[17]  Zhiying Wang,et al.  On multi-version coding for distributed storage , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[18]  Michael K. Reiter,et al.  Efficient Byzantine-tolerant erasure-coded storage , 2004, International Conference on Dependable Systems and Networks, 2004.

[19]  EE AlexanderSpiegelman,et al.  Space Bounds for Reliable Storage : Fundamental Limits of Coding , 2016 .