Neural Networks: More Than ‘Statistics for Amateurs’?

Statisticians often feel that neural networks are just an amateurish attempt to perform statistical inference, quite often using standard tools but always a non-standard terminology. We shed some light on this issue by looking at two leading cases of considerable overlap between statistics and neural networks, namely multilayer perceptrons for nonparametric regression and neural network-type on-line learning algorithms for extracting principal components.

[1]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[2]  Stephen D. Collins.  Neurocomputing 2 , 1992, Neurology.

[3]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[4]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[5]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[6]  T. Leen.  Dynamics of learning in linear feature-discovery networks , 1991 .

[7]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[8]  Sun-Yuan Kung,et al.  A neural network learning algorithm for adaptive principal component extraction (APEX) , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[9]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[10]  Jehoshua Bruck,et al.  Harmonic Analysis of Polynomial Threshold Functions , 1990, SIAM J. Discret. Math..

[11]  Brian D. Ripley,et al.  Statistical aspects of neural networks , 1993 .

[12]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[13]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[14]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[15]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[16]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[17]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[18]  James C. Bezdek,et al.  On the relationship between neural networks, pattern recognition and intelligence , 1992, Int. J. Approx. Reason..

[19]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[20]  Esther Levin,et al.  A statistical approach to learning and generalization in layered neural networks , 1989, Proc. IEEE.

[21]  Pierre Baldi,et al.  Linear Learning: Landscapes and Algorithms , 1988, NIPS.

[22]  James A. Anderson,et al.  Neurocomputing (vol. 2): directions for research , 1990 .

[23]  Kurt Hornik,et al.  Convergence analysis of local feature extraction algorithms , 1992, Neural Networks.

[24]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[25]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[26]  Kurt Hornik,et al.  Learning in linear neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[27]  Shun-Ichi Amari,et al.  Mathematical methods of neurocomputing , 1993 .

[28]  J. Rubner,et al.  A Self-Organizing Network for Principal-Component Analysis , 1989 .

[29]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[30]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[31]  Halbert White,et al.  Learning in Artificial Neural Networks: A Statistical Perspective , 1989, Neural Computation.

[32]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[33]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[34]  Arthur E. Bryson,et al.  Applied Optimal Control , 1969 .