MEASURING SPARSENESS OF NOISY SIGNALS

In this paper sparseness measures are reviewed, extended and compared. Special attention is paid on measuring sparseness of noisy data. We review and extend several definitions and measures for sparseness, including the � 0 , � p and � � norms. A measure based on order statistics is also proposed. The concept of sparseness is extended to the case where a signal has a dominant value other than zero. The sparseness measures can be easily modified to correspond to this new definition. Eight different measures are compared in three examples. It turns out that different measures may give complete opposite results if the distribution does not have a unique mode at zero. As conclusion, we suggest that the kurtosis should be avoided as a sparseness measure and recommend tanh-functions for measuring noisy sparseness.

[1]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[2]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[3]  R. Lambert.  Multichannel blind deconvolution: FIR matrix algebra and separation of multipath mixtures , 1996 .

[4]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[5]  F. Girosi,et al.  Sparse Correlation Kernel Analysis and Reconstruction , 1998 .

[6]  K. Jarrod Millman,et al.  Learning Sparse Codes with a Mixture-of-Gaussians Prior , 1999, NIPS.

[7]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[8]  R. Rosenfeld,et al.  Two decades of statistical language modeling: where do we go from here? , 2000, Proceedings of the IEEE.

[9]  Naoki Saito,et al.  Sparsity vs. statistical independence from a best-basis viewpoint , 2000, SPIE Optics + Photonics.

[10]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[11]  D. Donoho.  Sparse Components of Images and Optimal Atomic Decompositions , 2001 .

[12]  Joseph F. Murray,et al.  Dictionary Learning Algorithms for Sparse Representation , 2003, Neural Computation.

[13]  Bruno A. Olshausen,et al.  Principles of Image Representation in Visual Cortex , 2003 .

[14]  Arthur E. C. Pece,et al.  The Problem of Sparse Image Coding , 2002, Journal of Mathematical Imaging and Vision.