Neural Coding by Redundancy Reduction and Correlation

Redundancy reduction as a form of neural coding has been a topic of large research interest. A number of strategies has been proposed, but the one which is attracting the most attention assumes that this coding is carried out so that the output signals are mutually independent. In this work we go one step further and suggest an algorithm that separates also non-orthogonal signals (i.e. dependent signals). The resulting algorithm is very simple, as it is computationally economical and based on second order statistics that, as it is well know, is more robust to errors than higher order statistics. Moreover, the permutation/scaling problem is also avoided. The framework is given with a biological background, and we point out that the algorithm can also be used in other applications such as biomedical engineering and telecommunications.

[1]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[2]  A. J. Bell,et al.  Blind Separation of Event-Related Brain Responses into Independent Components , 1996 .

[3]  John Price-Wilkin,et al.  Oxford English Dictionary (2nd ed.) , 1991 .

[4]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[5]  P. Latham,et al.  Retinal ganglion cells act largely as independent encoders , 2001, Nature.

[6]  John Daugman,et al.  An information-theoretic view of analog representation in striate cortex , 1993 .

[7]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[8]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[9]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[10]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[11]  S Makeig,et al.  Blind separation of auditory event-related brain responses into independent components. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[13]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[14]  Andrzej Cichocki,et al.  New learning algorithm for blind separation of sources , 1992 .

[15]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[16]  Shun-ichi Amari,et al.  Adaptive blind signal processing-neural network approaches , 1998, Proc. IEEE.

[17]  H. B. Barlow,et al.  Unsupervised Learning , 1989, Neural Computation.

[18]  J. Simpson.  The Oxford English Dictionary and Its Philosophy , 2013 .