Recursive principal components analysis

A recurrent linear network can be trained with Oja's constrained Hebbian learning rule. As a result, the network learns to represent the temporal context associated to its input sequence. The operation performed by the network is a generalization of Principal Components Analysis (PCA) to time-series, called Recursive PCA. The representations learned by the network are adapted to the temporal statistics of the input. Moreover, sequences stored in the network may be retrieved explicitly, in the reverse order of presentation, thus providing a straight-forward neural implementation of a logical stack.

[1]  J. Rubner,et al.  Development of feature detectors by self-organization , 2004, Biological Cybernetics.

[2]  Cristopher Moore,et al.  Dynamical Recognizers: Real-Time Language Recognition by Analog Computers , 1998, Theor. Comput. Sci..

[3]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[4]  Risto Miikkulainen.  Subsymbolic Case-Role Analysis of Sentences with Embedded Clauses , 1993 .

[5]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[6]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[7]  Henry Markram,et al.  Temporal integration in recurrent microcircuits , 2003 .

[8]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[9]  Anil K. Jain,et al.  Artificial neural networks for feature extraction and multivariate data projection , 1995, IEEE Trans. Neural Networks.

[10]  José Carlos Príncipe,et al.  Robust on-line Principal Component Analysis based on a fixed-point approach , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[12]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[13]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[14]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[15]  Peter Ford Dominey,et al.  Linear recursive distributed representations , 2005, Neural Networks.

[16]  Benoît Champagne,et al.  Adaptive eigendecomposition of data covariance matrices based on first-order perturbations , 1994, IEEE Trans. Signal Process..

[17]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[18]  Thomas Voegtlin.  Learning principal components in a contextual space , 2000, ESANN.

[19]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[20]  K. Abed-Meraim,et al.  Natural power method for fast subspace tracking , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[21]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[22]  Deniz Erdogmus,et al.  Simultaneous Principal-Component Extraction with Application to Adaptive Blind Multiuser Detection , 2002, EURASIP J. Adv. Signal Process..

[23]  Sridha Sridharan,et al.  Multichannel speech separation by eigendecomposition and its application to co-talker interference removal , 1997, IEEE Trans. Speech Audio Process..

[24]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[25]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[26]  Masahiko Morita,et al.  Associative memory with nonmonotone dynamics , 1993, Neural Networks.

[27]  S.Y. Kung,et al.  Adaptive Principal component EXtraction (APEX) and applications , 1994, IEEE Trans. Signal Process..

[28]  C. Lee Giles,et al.  The Neural Network Pushdown Automaton: Architecture, Dynamics and Training , 1997, Summer School on Neural Networks.

[29]  Michel Cosnard,et al.  Computability with Low-Dimensional Dynamical Systems , 1994, Theor. Comput. Sci..

[30]  J. Rubner,et al.  A Self-Organizing Network for Principal-Component Analysis , 1989 .

[31]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[32]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[33]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..