A reliable LZ data compressor on reconfigurable coprocessors

Data compression techniques based on the Lempel-Ziv (LZ) algorithm are widely used in a variety of applications, especially in communications and data storage. However, since the LZ algorithm involves a considerable amount of parallel comparisons, it may be difficult to achieve a very high throughput using software approaches on general-purpose processors. In addition, error propagation due to single-bit transient errors during LZ compression causes a data integrity problem. We present an implementation of LZ data compression on reconfigurable hardware with concurrent error detection for high performance and reliability. Our approach achieves 100 Mbps throughput using four Xilinx 4036XLA FPGA chips. We also present an inverse comparison technique for LZ compression to guarantee data integrity with less area overhead than traditional systems based on duplication. The resulting execution time overhead and compression ratio degradation due to concurrent error detection is also minimized.

[1]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[2]  Jorma Rissanen,et al.  Generalized Kraft Inequality and Arithmetic Coding , 1976, IBM J. Res. Dev..

[3]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[4]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[5]  Mark Nelson,et al.  The Data Compression Book , 2009 .

[6]  Nagarajan Ranganathan,et al.  High-speed VLSI designs for Lempel-Ziv-based data compression , 1993 .

[7]  C.-Y. Lee,et al.  High-throughput data compressor designs using content addressable memory , 1995 .

[8]  Mark Nelson,et al.  The Data Compression Book, 2nd Edition , 1996 .

[9]  Dinesh Bhatia,et al.  Partial reconfiguration of FPGA mapped designs with applications to fault tolerance and yield enhancement , 1997, FPL.

[10]  Jonathan Rose,et al.  Partial Reconfiguration of FPGA Mapped Designs with Applications to Fault Tolerance and Yield Enhancement , 1997 .

[11]  David J. Craft,et al.  A fast hardware data compression algorithm and some algorithmic extensions , 1998, IBM J. Res. Dev..

[12]  Edward J. McCluskey,et al.  Dependable adaptive computing systems-the ROAR project , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[13]  Shantanu Dutt,et al.  Methodologies for Tolerating Cell and Interconnect Faults in FPGAs , 1998, IEEE Trans. Computers.

[14]  Edward J. McCluskey,et al.  Fault Location in FPGA-Based Reconfigurable Systems , 1998 .

[15]  Wayne P. Burleson,et al.  Efficient VLSI for Lempel-Ziv compression in wireless data communication networks , 1998, IEEE Trans. Very Large Scale Integr. Syst..

[16]  Nur A. Touba,et al.  A low cost approach for detecting, locating, and avoiding interconnect faults in FPGA-based reconfigurable systems , 1999, Proceedings Twelfth International Conference on VLSI Design. (Cat. No.PR00013).

[17]  Edward J. McCluskey,et al.  Dependable Computing and Online Testing in Adaptive and Configurable Systems , 2000, IEEE Des. Test Comput..