Wavelet-Like Receptive Fields Emerges by Non-Linear Minimization of Neuron Error

Redundancy reduction as a form of neural coding has been since the early sixties a topic of large research interest. A number of strategies has been proposed, but the one which is attracting most attention recently assumes that this coding is carried out so that the output signals are mutually independent. In this work we go one step further and suggest an strategy to deal also with non-orthogonal signals (i.e., "dependent" signals). Moreover, instead of working with the usual squared error, we design a neuron where the non-linearity is operating on the error. It is computationally more economic and, importantly, the permutation/scaling problem is avoided. The framework is given with a biological background, as we avocate throughout the manuscript that the algorithm fits well the single neuron and redundancy reduction doctrine. Moreover, we show that wavelet-like receptive fields emerges from natural images processed by this algorithm.

[1]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[2]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[3]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[4]  T. Shibasaki,et al.  Retinal ganglion cells act largely as independent encoders , 2001 .

[5]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[6]  Shun-ichi Amari,et al.  Adaptive blind signal processing-neural network approaches , 1998, Proc. IEEE.

[7]  Terrence J. Sejnowski,et al.  Blind separation and blind deconvolution: an information-theoretic approach , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[8]  Andrzej Cichocki,et al.  New learning algorithm for blind separation of sources , 1992 .

[9]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[10]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[11]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[12]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[13]  S Makeig,et al.  Blind separation of auditory event-related brain responses into independent components. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[14]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[15]  A. J. Bell,et al.  Blind Separation of Event-Related Brain Responses into Independent Components , 1996 .

[16]  John Daugman,et al.  An information-theoretic view of analog representation in striate cortex , 1993 .

[17]  J. Murray.  The Oxford English Dictionary , 1913 .