Input Feature Extraction for Multilayered Perceptrons Using Supervised Principal Component Analysis

A method is proposed for constructing salient features from a set of features that are given as input to a feedforward neural network used for supervised learning. Combinations of the original features are formed that maximize the sensitivity of the network's outputs with respect to variations of its inputs. The method exhibits some similarity to Principal Component Analysis, but also takes into account supervised character of the learning task. It is applied to classification problems leading to improved generalization ability originating from the alleviation of the curse of dimensionality problem.

[1]  Richard S. Johannes,et al.  Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus , 1988 .

[2]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[3]  Juan-Manuel Torres-Moreno,et al.  Characterization of the Sonar Signals Benchmark , 1998, Neural Processing Letters.

[4]  Kevin Baker,et al.  Classification of radar returns from the ionosphere using neural networks , 1989 .

[5]  Sun-Yuan Kung,et al.  A neural network learning algorithm for adaptive principal component extraction (APEX) , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[6]  P. Foldiak,et al.  Adaptive network for optimal linear feature extraction , 1989, International 1989 Joint Conference on Neural Networks.

[7]  Dimitris A. Karras,et al.  An efficient constrained learning algorithm with momentum acceleration , 1995, Neural Networks.

[8]  M. Kendall,et al.  Kendall's advanced theory of statistics , 1995 .

[9]  Kenneth W. Bauer,et al.  Determining input features for multilayer perceptrons , 1995, Neurocomputing.

[10]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[11]  R. Fisher.  The Advanced Theory of Statistics , 1943, Nature.

[12]  R. Preisendorfer,et al.  Principal Component Analysis in Meteorology and Oceanography , 1988 .

[13]  Wright-Patterson Afb,et al.  Feature Selection Using a Multilayer Perceptron , 1990 .

[14]  Stavros J. Perantonis,et al.  Efficient Linear Discriminant Analysis Using a Fast Quadratic Programming Algorithm , 1998 .

[15]  I. Jolliffe.  Principal Component Analysis , 2002 .

[16]  G. L. Tarr,et al.  Multi-layered feedforward neural networks for image segmentation , 1992 .

[17]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[18]  R. Liu,et al.  Adaptive distributed orthogonalization processing for principal components analysis , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[20]  Kari Karhunen,et al.  Über lineare Methoden in der Wahrscheinlichkeitsrechnung , 1947 .

[21]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.