Error Entropy and Mean Square Error Minimization for Lossless Image Compression

In this paper, the minimum error entropy (MEE) criterion is considered as an alternative to the mean square error (MSE) criterion in obtaining predictor coefficients for lossless still image coding. Estimation of the error entropy is done using Renyi's formula. The PDF of the error between image pixels and the predicted values is estimated using the Parzen windowing with a Gaussian kernel. The performance of the error entropy minimization and the mean square error minimization is compared using the first order Shannon's entropy of the residual error. Comparison between MEE and MSE is extended to the issue of treating the image as a number of independent blocks, where each block uses its optimized predictor. The behavior of MEE is similar to MSE with a small improvement when using the maximum allowable window size.

[1]  F. Harris.  On the use of windows for harmonic analysis with the discrete Fourier transform , 1978, Proceedings of the IEEE.

[2]  D. Erdogmus,et al.  Nonlinear channel equalization using multilayer perceptrons with information-theoretic criterion , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).

[3]  Kuldip K. Paliwal,et al.  Minimum-entropy clustering and its application to lossless image coding , 1997, Proceedings of International Conference on Image Processing.

[4]  D. Erdogmus,et al.  Entropy minimization algorithm for multilayer perceptrons , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[5]  Alfréd Rényi,et al.  Probability Theory , 1970 .

[6]  J. FREDRIC.  On the Use of Windows for Harmonic Analysis with the Discrete Fourier Transform , .

[7]  Deniz Erdogmus,et al.  Convergence properties and data efficiency of the minimum error entropy criterion in ADALINE training , 2003, IEEE Trans. Signal Process..

[8]  Deniz Erdoğmuş,et al.  COMPARISON OF ENTROPY AND MEAN SQUARE ERROR CRITERIA IN ADAPTIVE SYSTEM TRAINING USING HIGHER ORDER STATISTICS , 2004 .

[9]  Khalid Sayood,et al.  Introduction to Data Compression , 1996 .

[10]  Deniz Erdogmus,et al.  Entropy minimization for supervised digital communications channel equalization , 2002, IEEE Trans. Signal Process..