MDS Code Constructions With Small Sub-Packetization and Near-Optimal Repair Bandwidth

This paper addresses the problem of constructing maximum distance separable (MDS) codes that enable exact reconstruction (repair) of each code block by downloading a small amount of information from the remaining code blocks. The total amount of information flow from the remaining code blocks during this reconstruction process is referred to as repair bandwidth of the underlying code. Existing constructions of exact-repairable MDS codes with optimal repair bandwidth require working with large subpacketization levels, which restrict their applicability in practice. This paper presents two general approaches to construct exact-repairable MDS codes that aim at significantly reducing the required subpacketization level at the cost of slightly suboptimal repair bandwidth. The first approach provides MDS codes that have repair bandwidth at most twice the optimal repair bandwidth. In addition, these codes also have the smallest possible subpacketization level <inline-formula> <tex-math notation="LaTeX">$O(r)$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$r$ </tex-math></inline-formula> denotes the number of parity blocks. This approach is then generalized to design codes that have their repair bandwidth approaching the optimal repair bandwidth at the cost of graceful increment in the required subpacketization level. The second approach transforms an MDS code with optimal repair bandwidth and large subpacketization level into a longer MDS code with small subpacketization level and near-optimal repair bandwidth. For a given <inline-formula> <tex-math notation="LaTeX">$r$ </tex-math></inline-formula>, the codes constructed using this approach have their subpacketization level scaling logarithmically with the code length. In addition, the obtained codes require field size only linear in the code length and ensure load balancing among the intact code blocks in terms of the information downloaded from these blocks during the exact reconstruction of a code block.

[1]  Kenneth W. Shum,et al.  Cooperative Regenerating Codes , 2012, IEEE Transactions on Information Theory.

[2]  Venkatesan Guruswami,et al.  Repairing Reed-Solomon Codes , 2015, IEEE Transactions on Information Theory.

[3]  John C. S. Lui,et al.  Single Disk Failure Recovery for X-Code-Based Parallel Storage Systems , 2014, IEEE Transactions on Computers.

[4]  P. Vijay Kumar,et al.  A high-rate MSR code with polynomial sub-packetization level , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[5]  Alexander Barg,et al.  Explicit Constructions of High-Rate MDS Array Codes With Optimal Repair Bandwidth , 2016, IEEE Transactions on Information Theory.

[6]  Cheng Huang,et al.  On the Locality of Codeword Symbols , 2011, IEEE Transactions on Information Theory.

[7]  Venkatesan Guruswami,et al.  ∊-MSR codes with small sub-packetization , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[8]  Dimitris S. Papailiopoulos,et al.  Locally Repairable Codes , 2012, IEEE Transactions on Information Theory.

[9]  Kannan Ramchandran,et al.  A Piggybacking Design Framework for Read-and Download-Efficient Distributed Storage Codes , 2013, IEEE Transactions on Information Theory.

[10]  Nihar B. Shah,et al.  Optimal Exact-Regenerating Codes for Distributed Storage at the MSR and MBR Points via a Product-Matrix Construction , 2010, IEEE Transactions on Information Theory.

[11]  Hoang Dau,et al.  Optimal repair schemes for some families of full-length reed-solomon codes , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[12]  Kannan Ramchandran,et al.  Asymptotic Interference Alignment for Optimal Repair of MDS Codes in Distributed Storage , 2013, IEEE Transactions on Information Theory.

[13]  Jehoshua Bruck,et al.  Long MDS codes for optimal repair bandwidth , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[14]  Jehoshua Bruck,et al.  Access Versus Bandwidth in Codes for Storage , 2014, IEEE Transactions on Information Theory.

[15]  Anne-Marie Kermarrec,et al.  Repairing Multiple Failures with Coordinated and Adaptive Regenerating Codes , 2011, 2011 International Symposium on Networking Coding.

[16]  Alexandros G. Dimakis,et al.  Rebuilding for array codes in distributed storage systems , 2010, 2010 IEEE Globecom Workshops.

[17]  P. Vijay Kumar,et al.  Codes With Local Regeneration and Erasure Correction , 2014, IEEE Transactions on Information Theory.

[18]  Cheng Huang,et al.  Polynomial length MDS codes with optimal repair in distributed storage , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[19]  Sriram Vishwanath,et al.  Centralized Repair of Multiple Node Failures With Applications to Communication Efficient Secret Sharing , 2016, IEEE Transactions on Information Theory.

[20]  F. MacWilliams,et al.  The Theory of Error-Correcting Codes , 1977 .

[21]  P. Vijay Kumar,et al.  An Explicit, Coupled-Layer Construction of a High-Rate MSR Code with Low Sub-Packetization Level, Small Field Size and All-Node Repair , 2016, ArXiv.

[22]  Jehoshua Bruck,et al.  Zigzag Codes: MDS Array Codes With Optimal Rebuilding , 2011, IEEE Transactions on Information Theory.

[23]  Alexander Barg,et al.  Explicit Constructions of Optimal-Access MDS Codes With Nearly Optimal Sub-Packetization , 2016, IEEE Transactions on Information Theory.

[24]  Itzhak Tamo,et al.  A Family of Optimal Locally Recoverable Codes , 2013, IEEE Transactions on Information Theory.

[25]  Sachin Katti,et al.  Copysets: Reducing the Frequency of Data Loss in Cloud Storage , 2013, USENIX Annual Technical Conference.

[26]  Sriram Vishwanath,et al.  Optimal Locally Repairable and Secure Codes for Distributed Storage Systems , 2012, IEEE Transactions on Information Theory.

[27]  Jehoshua Bruck,et al.  Explicit Minimum Storage Regenerating Codes , 2016, IEEE Transactions on Information Theory.

[28]  Dimitris S. Papailiopoulos,et al.  Repair optimal erasure codes through hadamard designs , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[29]  Cheng Huang,et al.  Explicit Maximally Recoverable Codes With Locality , 2013, IEEE Transactions on Information Theory.

[30]  Dimitris S. Papailiopoulos,et al.  A repair framework for scalar MDS codes , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[31]  Arman Fazeli,et al.  Minimum storage regenerating codes for all parameters , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[32]  Wentao Huang,et al.  Communication Efficient Secret Sharing , 2015, IEEE Transactions on Information Theory.

[33]  Han Mao Kiah,et al.  Repairing Reed-Solomon Codes With Multiple Erasures , 2016, IEEE Transactions on Information Theory.

[34]  Harald Øverby,et al.  HashTag Erasure Codes: From Theory to Practice , 2016, IEEE Transactions on Big Data.

[35]  Harald Øverby,et al.  General Sub-Packetized Access-Optimal Regenerating Codes , 2016, IEEE Communications Letters.

[36]  A. Robert Calderbank,et al.  An Improved Sub-Packetization Bound for Minimum Storage Regenerating Codes , 2013, IEEE Transactions on Information Theory.

[37]  Sriram Vishwanath,et al.  Progress on high-rate MSR codes: Enabling arbitrary number of helper nodes , 2016, 2016 Information Theory and Applications Workshop (ITA).

[38]  Cory Hill,et al.  f4: Facebook's Warm BLOB Storage System , 2014, OSDI.

[39]  Mario Blaum,et al.  Partial-MDS Codes and Their Application to RAID Type of Architectures , 2012, IEEE Transactions on Information Theory.

[40]  Alexandros G. Dimakis,et al.  Network Coding for Distributed Storage Systems , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[41]  Yunnan Wu,et al.  A Survey on Network Codes for Distributed Storage , 2010, Proceedings of the IEEE.

[42]  Itzhak Tamo,et al.  Optimal Repair of Reed-Solomon Codes: Achieving the Cut-Set Bound , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[43]  Alexander Barg,et al.  Explicit constructions of MDS array codes and RS codes with optimal repair bandwidth , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[44]  Cheng Huang,et al.  Rethinking erasure codes for cloud file systems: minimizing I/O for recovery and degraded reads , 2012, FAST.

[45]  Venkatesan Guruswami,et al.  MDS code constructions with small sub-packetization and near-optimal repair bandwidth , 2017, SODA 2017.

[46]  Mary Wootters,et al.  Repairing multiple failures for scalar MDS codes , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[47]  Jehoshua Bruck,et al.  On codes for optimal rebuilding access , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).