Generalized Optimal Storage Scaling via Network Coding

It is critical to support efficient scaling in distributed storage systems so as to meet increasing storage demands with new storage nodes. However, the scaling process incurs substantial scaling bandwidth due to reorganizing currently stored data to new storage nodes. Recent work has applied network coding to minimize scaling bandwidth for a special case where <tex>$(n,\ k)$</tex> MDS codes are scaled to <tex>$(n^{\prime}, k^{\prime})$</tex> MDS codes for <tex>$n^{\prime}-k^{\prime}=n-k$</tex>. In this paper, we extend the results and prove the minimum scaling bandwidth for a more general setting where <tex>$(n,\ k)$</tex> MDS codes are scaled to <tex>$(n^{\prime}, k^{\prime})$</tex> MDS codes for <tex>$n^{\prime} > n$</tex> and <tex>$k^{\prime}\geq k$</tex>. Furthermore, we present a family of MDS code construction that achieves optimal scaling from <tex>$(n,\ k)$</tex> to <tex>$(n^{\prime}, k^{\prime})$</tex> where <tex>$k=k^{\prime}$</tex>.

[1]  Weimin Zheng,et al.  FastScale: Accelerate RAID Scaling by Minimizing Data Migration , 2011, FAST.

[2]  Si Wu,et al.  I/O-Efficient Scaling Schemes for Distributed Storage Systems with CRS Codes , 2016, IEEE Transactions on Parallel and Distributed Systems.

[3]  Rajeev Motwani,et al.  Randomized Algorithms , 1995, SIGA.

[4]  Rudolf Ahlswede,et al.  Network information flow , 2000, IEEE Trans. Inf. Theory.

[5]  Xiao Qin,et al.  Scale-RS: An Efficient Scaling Scheme for RS-Coded Storage Clusters , 2015, IEEE Transactions on Parallel and Distributed Systems.

[6]  Chentao Wu,et al.  GSR: A Global Stripe-Based Redistribution Approach to Accelerate RAID-5 Scaling , 2012, 2012 41st International Conference on Parallel Processing.

[7]  Brijesh Kumar Rai,et al.  On adaptive distributed storage systems , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[8]  Cheng Huang,et al.  Erasure Coding in Windows Azure Storage , 2012, USENIX Annual Technical Conference.

[9]  Pan Zhou,et al.  Toward Optimal Storage Scaling via Network Coding: From Theory to Practice , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[10]  Van-Anh Truong,et al.  Availability in Globally Distributed Storage Systems , 2010, OSDI.

[11]  Alexandros G. Dimakis,et al.  Network Coding for Distributed Storage Systems , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[12]  James S. Plank A tutorial on Reed-Solomon coding for fault-tolerance in RAID-like systems , 1997 .