NATURE VS . MATH : INTERPRETING INDEPENDENT COMPONENT ANALYSIS IN LIGHT OF COMPUTATIONAL HARMONIC ANALYSIS

ICA has recently been applied to naturally-occurring data to uncover hidden fundamental components. Harmonic analysis has long been used to uncover hidden fundamental components of mathematically-defined objects. In my talk I will explore some recent parallelisms between ICA and CHA – fascinating similarities in the “hidden components” that the two subjects are uncovering. I will suggest that recent work on the independent components of images can best be interepreted in the light of recent constructions in harmonic analysis – such as ridgelets and curvelets. These mathematical objects exhibit surprising similarities to the results of ICA on certain naturally-occurring data.

[1]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[2]  D. Donoho.  ON MINIMUM ENTROPY DECONVOLUTION , 1981 .

[3]  H. B. Barlow,et al.  What does the eye see best? , 1983, Nature.

[4]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[5]  Leslie S. Smith,et al.  The principal components of natural images , 1992 .

[6]  Yves Meyer,et al.  Wavelets and Applications , 1992 .

[7]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[8]  D. Ruderman.  The statistics of natural images , 1994 .

[9]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[10]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[11]  C. Fyfe,et al.  Finding compact and sparse-distributed representations of visual images , 1995 .

[12]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[13]  R W Prager,et al.  Development of low entropy coding in a recurrent network. , 1996, Network.

[14]  Aapo Hyvrinen.  Independent Component Analysis by Minimization of Mutual Information Independent Component Analysis by Minimization of Mutual Information Independent Component Analysis by Minimization of Mutual Information , 1997 .

[15]  Erkki Oja,et al.  Applications of neural blind separation to signal and image processing , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  Bruno A. Olshausen,et al.  Inferring Sparse, Overcomplete Image Codes Using an Efficient Coding Framework , 1998, NIPS.

[17]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[18]  D. Ruderman,et al.  INDEPENDENT COMPONENT ANALYSIS OF NATURAL IMAGE SEQUENCES YIELDS SPATIOTEMPORAL FILTERS SIMILAR TO SIMPLE CELLS IN PRIMARY VISUAL CORTEX , 1998 .

[19]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[20]  E. Candès.  Harmonic Analysis of Neural Networks , 1999 .

[21]  D. Donoho,et al.  Tight frames of k-plane ridgelets and the problem of representing objects that are smooth away from d-dimensional singularities in Rn. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[23]  D. Donoho.  Wedgelets: nearly minimax estimation of edges , 1999 .

[24]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[25]  B. Silverman,et al.  Functional Data Analysis , 1997 .

[26]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[27]  David L. Donoho,et al.  Orthonormal Ridgelets and Linear Singularities , 2000, SIAM J. Math. Anal..