Lazy Management for Frequency Table on Hardware-Based Stream Lossless Data Compression

The demand for communicating large amounts of data in real-time has raised new challenges with implementing high-speed communication paths for high definition video and sensory data. It requires the implementation of high speed data paths based on hardware. Implementation difficulties have to be addressed by applying new techniques based on data-oriented algorithms. This paper focuses on a solution for this problem by applying a lossless data compression mechanism on the communication data path. The new lossless data compression mechanism, called LCA-DLT, provides dynamic histogram management for symbol lookup tables used in the compression and the decompression operations. When the histogram memory is fully used, the management algorithm needs to find the least used entries and invalidate these entries. The invalidation operations cause the blocking of the compression and the decompression data stream. This paper proposes novel techniques to eliminate blocking by introducing a dynamic invalidation mechanism, which allows achievement of a high throughput data compression.

[1]  Hiroshi Sakamoto,et al.  Stream-Based Lossless Data Compression Hardware Using Adaptive Frequency Table Management , 2015, BPOE.

[2]  Joo-Young Kim,et al.  A Scalable Multi-engine Xpress9 Compressor with Asynchronous Data Transfer , 2014, 2014 IEEE 22nd Annual International Symposium on Field-Programmable Custom Computing Machines.

[3]  Divyakant Agrawal,et al.  An integrated efficient solution for computing frequent and top-k elements in data streams , 2006, TODS.

[4]  Koichi Wada,et al.  Performance Enhancement of Inter-Cluster Communication with Software-based Data Compression Link Layer , 2005, IASTED PDCS.

[5]  Joo-Young Kim,et al.  A Scalable High-Bandwidth Architecture for Lossless Compression on FPGAs , 2015, 2015 IEEE 23rd Annual International Symposium on Field-Programmable Custom Computing Machines.

[6]  Panos Nasiopoulos,et al.  Efficient Lossless Compression of 4-D Medical Images Based on the Advanced Video Coding Scheme , 2008, IEEE Transactions on Information Technology in Biomedicine.

[7]  David A. Wood,et al.  Adaptive cache compression for high-performance processors , 2004, Proceedings. 31st Annual International Symposium on Computer Architecture, 2004..

[8]  E. Jebamalar Leavline,et al.  Hardware Implementation of LZMA Data Compression Algorithm , 2013 .

[9]  Liang-Gee Chen,et al.  Architecture Design of Fine Grain Quality Scalable Encoder with CABAC for H.264/AVC Scalable Extension , 2010, J. Signal Process. Syst..

[10]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Chin-Chen Chang,et al.  Efficient reversible data hiding for VQ-compressed images based on index mapping mechanism , 2013, Signal Process..

[12]  Xi Chen,et al.  C-Pack: A High-Performance Microprocessor Cache Compression Algorithm , 2010, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[13]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[14]  Joan L. Mitchell,et al.  Optimal Hardware and Software Arithmetic Coding Procedures for the Q-Coder , 1988, IBM J. Res. Dev..

[15]  Per Stenström,et al.  SC2: A statistical compression cache scheme , 2014, 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA).

[16]  Jorma Rissanen,et al.  Compression of Black-White Images with Arithmetic Coding , 1981, IEEE Trans. Commun..

[17]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[18]  Michael E. Wazlowski,et al.  IBM Memory Expansion Technology (MXT) , 2001, IBM J. Res. Dev..

[19]  Chin-Chen Chang,et al.  A Novel Joint Data-Hiding and Compression Scheme Based on SMVQ and Image Inpainting , 2014, IEEE Transactions on Image Processing.

[20]  Javier D. Bruguera,et al.  Arithmetic coding architecture for H.264/AVC CABAC compression system , 2004 .

[21]  Jeffrey Scott Vitter,et al.  Design and analysis of dynamic Huffman codes , 1987, JACM.

[22]  Prasant Mohapatra,et al.  Hardware Architecture for Simultaneous Arithmetic Coding and Encryption , 2011 .

[23]  Marcin Zukowski,et al.  Super-Scalar RAM-CPU Cache Compression , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[24]  Richard M. Karp,et al.  A simple algorithm for finding frequent elements in streams and bags , 2003, TODS.

[25]  Onur Mutlu,et al.  Base-delta-immediate compression: Practical data compression for on-chip caches , 2012, 2012 21st International Conference on Parallel Architectures and Compilation Techniques (PACT).

[26]  S. Pothula,et al.  C-Pack : A High-Performance Microprocessor Cache Compression Algorithm , 2011 .

[27]  Hong Ren Wu,et al.  Perceptually lossless medical image coding , 2006, IEEE Transactions on Medical Imaging.

[28]  Hiroshi Sakamoto,et al.  An Online Algorithm for Lightweight Grammar-Based Compression , 2011, 2011 First International Conference on Data Compression, Communications and Processing.

[29]  Ming-Der Shieh,et al.  Reconfigurable architecture for entropy decoding and inverse transform in H.264 , 2010, IEEE Transactions on Consumer Electronics.