Feature Extraction using an Unsupervised Neural Network

A novel unsupervised neural network for dimensionality reduction which seeks directions emphasizing distinguishing features in the data is presented. A statistical framework for the parameter estimation problem associated with this neural network is given and its connection to exploratory projection pursuit methods is established. The network is shown to minimize a loss function (projection index) over a set of parameters, yielding an optimal decision rule under some norm. A specific projection index that favors directions possessing multimodality is presented. This leads to a similar form to the synaptic modification equations governing learning in Bienenstock, Cooper, and Munro (BCM) neurons (1982). The importance of a dimensionality reduction principal based, solely on distinguishing features, is demonstrated using a linguistically motivated phoneme recognition experiment, and compared with feature extraction using principal components and back-propagation network.

[1]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[2]  P. Hall.  Estimating the direction in which a data set is most interesting , 1988 .

[3]  L. Cooper,et al.  A physiological basis for a theory of synapse modification. , 1987, Science.

[4]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[5]  J. Friedman,et al.  PROJECTION PURSUIT DENSITY ESTIMATION , 1984 .

[6]  P. Hall.  On Polynomial-Based Projection Indices for Exploratory Projection Pursuit , 1989 .

[7]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[8]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[9]  Richard Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[10]  J. Kruskal.  TOWARD A PRACTICAL METHOD WHICH HELPS UNCOVER THE STRUCTURE OF A SET OF MULTIVARIATE OBSERVATIONS BY FINDING THE LINEAR TRANSFORMATION WHICH OPTIMIZES A NEW “INDEX OF CONDENSATION” , 1969 .

[11]  L N Cooper,et al.  Mean-field theory of a neural network. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Raymond M. Wright,et al.  Numerical classification applied to certain Jamaican eocene nummulitids , 1971 .

[13]  D. Freedman,et al.  Asymptotics of Graphical Projection Pursuit , 1984 .

[14]  E. Zwicker,et al.  Subdivision of the audible frequency range into critical bands , 1961 .

[15]  Leon N. Cooper,et al.  Development and properties of neural networks , 1985 .