Enhanced LZMA and BZIP2 for improved energy data compression

Smart grid is the next generation of electricity production, transmission and distribution system. This is possible through an overlayed communication layer with the power delivery layer. Due to this communication layer smart grids produce enormous amounts of data. This data may be analyzed for improving the quality of service of smart grids. However, handling such enormous amount of data is a challenge. LZMA and BZIP2 are two industrial strength compression techniques. In this paper we present an enhanced version of these two schemes specifically targeted to smart grid data through a pre-processing step. Our results show that while the original LZMA is able to compress the data size to around 80% our enhanced scheme using the preprocessing is able to reduce the size of the smart grid data to 98% on average.

[1]  Jan Kraus,et al.  LOOSELESS ENCODINGS AND COMPRESSION ALGORITHMS APPLIED ON POWER QUALITY DATASETS , 2009 .

[2]  R. Nigel Horspool,et al.  Data Compression Using Dynamic Markov Modelling , 1987, Comput. J..

[3]  Alistair Moffat,et al.  Implementing the PPM data compression scheme , 1990, IEEE Trans. Commun..

[4]  Ömer Nezih Gerek,et al.  Compression of power quality event data using 2D representation , 2008 .

[5]  E. Michael Azoff,et al.  Neural Network Time Series: Forecasting of Financial Markets , 1994 .

[6]  P.A. Crossley,et al.  Bridging the gap between signal and power , 2009, IEEE Signal Processing Magazine.

[7]  A. Vojdani,et al.  Smart Integration , 2008, IEEE Power and Energy Magazine.

[8]  Damir Novosel,et al.  Grids get smart protection and control , 1997 .

[9]  Irene Yu-Hua Gu,et al.  Bridge the gap: signal processing for power quality applications , 2003 .

[10]  Farrokh Albuyeh,et al.  Grid of the future , 2009, IEEE Power and Energy Magazine.

[11]  J. Zico Kolter,et al.  REDD : A Public Data Set for Energy Disaggregation Research , 2011 .

[12]  B.F. Wollenberg,et al.  Toward a smart grid: power delivery for the 21st century , 2005, IEEE Power and Energy Magazine.

[13]  Paulo F. Ribeiro,et al.  Trends, Challenges and Opportunities in Power Quality Research , 2010 .

[14]  Irene Y. H. Gu,et al.  Emerging Signal Processing Techniques for Power Quality Applications , 2007, EURASIP J. Adv. Signal Process..

[15]  João Marcos Travassos Romano,et al.  The Compression of Electric Signal Waveforms for Smart Grids: State of the Art and Future Trends , 2014, IEEE Transactions on Smart Grid.