On the Use of Projection Pursuit Constraints for Training Neural Networks

We present a novel classification and regression method that combines exploratory projection pursuit (unsupervised training) with projection pursuit regression (supervised training), to yield a new family of cost/complexity penalty terms. Some improved generalization properties are demonstrated on real world problems.

[1]  Nathan Intrator,et al.  Objective function formulation of the BCM theory of visual cortical plasticity: Statistical connections, stability conditions , 1992, Neural Networks.

[2]  P. Hall.  On Projection Pursuit Regression , 1989 .

[3]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[4]  P. Hall.  On Polynomial-Based Projection Indices for Exploratory Projection Pursuit , 1989 .

[5]  Nathan Intrator,et al.  Three-Dimensional Object Recognition Using an Unsupervised BCM Network: The Usefulness of Distinguishing Features , 1993, Neural Computation.

[6]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[7]  Nathan Intrator,et al.  Supervised and unsupervised feature extraction from a cochlear model for speech recognition , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.

[8]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[9]  Nathan Intrator,et al.  Feature Extraction Using an Unsupervised Neural Network , 1992, Neural Computation.

[10]  J. Kruskal.  TOWARD A PRACTICAL METHOD WHICH HELPS UNCOVER THE STRUCTURE OF A SET OF MULTIVARIATE OBSERVATIONS BY FINDING THE LINEAR TRANSFORMATION WHICH OPTIMIZES A NEW “INDEX OF CONDENSATION” , 1969 .

[11]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[12]  Nathan Intrator,et al.  Phonetic classification of timit segments preprocessed with lyon's cochlear model using a supervised/unsupervised hybrid neural network , 1992, ICSLP.

[13]  P. Hall.  Estimating the direction in which a data set is most interesting , 1988 .

[14]  Robert Azencott,et al.  Image compression with back propagation: Improvement of the visual restoration using different cost functions , 1991, Neural Networks.

[15]  L. Cooper,et al.  Synaptic plasticity in visual cortex: comparison of theory with experiment. , 1991, Journal of neurophysiology.

[16]  Richard F. Lyon,et al.  A computational model of filtering, detection, and compression in the cochlea , 1982, ICASSP.

[17]  Josef Skrzypek,et al.  Synergy of Clustering Multiple Back Propagation Networks , 1989, NIPS.

[18]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[19]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[20]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  Nathan Intrator,et al.  Three-Dimensional Object Recognition Using an Unsupervised Neural Network: Understanding the Distinguishing Features , 1991 .

[22]  L. Jones.  On a conjecture of Huber concerning the convergence of projection pursuit regression , 1987 .

[23]  D. Freedman,et al.  Asymptotics of Graphical Projection Pursuit , 1984 .

[24]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[25]  Barak A. Pearlmutter,et al.  Chaitin-Kolmogorov Complexity and Generalization in Neural Networks , 1990, NIPS.

[26]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[27]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[28]  Peter Seitz,et al.  Minimum class entropy: A maximum information approach to layered networks , 1989, Neural Networks.

[29]  Nathan Intrator,et al.  Combining Exploratory Projection Pursuit and Projection Pursuit Regression with Application to Neural Networks , 1993, Neural Computation.