Data Compression Device Based on Modified LZ4 Algorithm

Data compression is commonly used in NAND flash-based solid state drives (SSDs) to increase their storage performance and lifetime as it can reduce the amount of data written to and read from NAND flash memory. Software-based data compression reduces SSD performance significantly and, as such, hardware-based data compression designs are required. This paper studies the latest lossless data compression algorithm, i.e., the Lempel-Ziv (LZ)4 algorithm which is one of the fastest compression algorithms reported to date. A data compression FPGA prototype based on the LZ4 lossless compression algorithm is studied. The original LZ4 compression algorithm is modified for real-time hardware implementation. Two hardware architectures of the modified LZ4 algorithm (MLZ4) are proposed with both compressors and decompressors, which are implemented on an FPGA evaluation kit. The implementation results show that the proposed compressor architecture can achieve a high throughput of up to 1.92 Gb/s with a compression ratio of up to 2.05, which is higher than all previous LZ algorithm designs implemented on FPGAs. The compression device can be used in high-end SSDs to further increase their storage performance and lifetime.

[1]  Sebastian Deorowicz,et al.  Universal lossless data compression algorithms , 2003 .

[2]  David Salomon,et al.  Data compression - The Complete Reference, 4th Edition , 2004 .

[3]  James A. Storer,et al.  Data compression via textual substitution , 1982, JACM.

[4]  Sungjin Lee,et al.  Improving performance and lifetime of solid-state drives using hardware-accelerated compression , 2011, IEEE Transactions on Consumer Electronics.

[5]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[6]  Sven Ubik,et al.  LZ4 compression algorithm on FPGA , 2015, 2015 IEEE International Conference on Electronics, Circuits, and Systems (ICECS).

[7]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[8]  Mark Nelson,et al.  The Data Compression Book, 2nd Edition , 1996 .

[9]  Jin-Soo Kim,et al.  zFTL: power-efficient data compression support for NAND flash-based consumer electronics devices , 2011, IEEE Transactions on Consumer Electronics.

[10]  Mark Nelson,et al.  The data compression book (2nd ed.) , 1995 .

[11]  Sivan Toledo,et al.  Compression and SSDs: Where and How? , 2014, INFLOW.

[12]  D. Huffman.  A Method for the Construction of Minimum-Redundancy Codes , 1952 .

[13]  Onur Mutlu,et al.  Error Characterization, Mitigation, and Recovery in Flash-Memory-Based Solid-State Drives , 2017, Proceedings of the IEEE.

[14]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[15]  Terry A. Welch,et al.  A Technique for High-Performance Data Compression , 1984, Computer.

[16]  Zhenhua Li,et al.  A solid state drive architecture with memory card modules , 2016, IEEE Transactions on Consumer Electronics.

[17]  Raja Ali Riaz,et al.  Optimized RTL design and implementation of LZW algorithm for high bandwidth applications , 2011 .

[18]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[19]  Habibullah Jamal,et al.  Multigig lossless data compression device , 2010, IEEE Transactions on Consumer Electronics.

[20]  Lin Zhang,et al.  Implementation of LZMA compression algorithm on FPGA , 2014 .

[21]  W. Cui.  New LZW Data Compression Algorithm and Its FPGA Implementation Author ( s ) : , 2007 .

[22]  K. L. Ho,et al.  Teletext data change detection and noiseless data compression , 1995 .

[23]  David Salomon,et al.  Data Compression: The Complete Reference , 2006 .

[24]  Danny Harnik,et al.  A Fast Implementation of Deflate , 2014, 2014 Data Compression Conference.

[25]  Ross N. Williams,et al.  An extremely fast Ziv-Lempel data compression algorithm , 1991, [1991] Proceedings. Data Compression Conference.

[26]  José Luis Núñez,et al.  Gbit/s lossless data compression hardware , 2003, IEEE Trans. Very Large Scale Integr. Syst..

[27]  Bing Li,et al.  Implementation of the LZMA compression algorithm on FPGA , 2017, 2017 International Conference on Electron Devices and Solid-State Circuits (EDSSC).

[28]  Sivan Toledo,et al.  Compression and SSD : Where and How ? , 2014 .