On the Optimal Structure Design of Multilayer Feedforward Neural Networks for Pattern Recognition

In this survey paper, the-state-of-the-art of the optimal structure design of Multilayer Feedforward Neural Network (MFNN) for pattern recognition is reviewed. Special emphasis is laid on the scale-limited MFNN and the internal representation and decision boundary-based design methodologies. A comprehensively comparative study of the main characteristics of each method is presented. Also, future research directions are outlined.

[1]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[2]  Guang-Bin Huang,et al.  Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions , 1998, IEEE Trans. Neural Networks.

[3]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[4]  Bernhard Schölkopf,et al.  Comparison of View-Based Object Recognition Algorithms Using Realistic 3D Models , 1996, ICANN.

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[7]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[8]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[9]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[10]  Yoshio Hirose,et al.  Backpropagation algorithm which varies the number of hidden units , 1989, International 1989 Joint Conference on Neural Networks.

[11]  Saul B. Gelfand,et al.  Classification trees with neural network feature extraction , 1992, IEEE Trans. Neural Networks.

[12]  James T. Kwok,et al.  Constructive algorithms for structure learning in feedforward neural networks for regression problems , 1997, IEEE Trans. Neural Networks.

[13]  David Lowe,et al.  The optimised internal representation of multilayer classifier networks performs nonlinear discriminant analysis , 1990, Neural Networks.

[14]  Chi-Sang Poon,et al.  Linear independence of internal representations in multilayer perceptrons , 1999, IEEE Trans. Neural Networks.

[15]  Ralf Salomon,et al.  Evolutionary algorithms and gradient search: similarities and differences , 1998, IEEE Trans. Evol. Comput..

[16]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[17]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[18]  Mahmood R. Azimi-Sadjadi,et al.  Recursive dynamic node creation in multilayer neural networks , 1993, IEEE Trans. Neural Networks.

[19]  David A. Landgrebe,et al.  Decision boundary feature extraction for neural networks , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[20]  Lawrence O. Hall,et al.  Divide and Conquer Neural Networks , 1993, Neural Networks.

[21]  Shin'ichi Tamura,et al.  Capabilities of a four-layered feedforward neural network: four layers versus three , 1997, IEEE Trans. Neural Networks.

[22]  Jürgen Schürmann,et al.  Pattern classification , 2008 .

[23]  Etsuji Tomita,et al.  Separability of internal representations in multilayer perceptrons with application to learning , 1993, Neural Networks.

[24]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[25]  David G. Lowe,et al.  Optimized Feature Extraction and the Bayes Decision in Feed-Forward Classifier Networks , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[27]  Anil K. Jain,et al.  Guest Editorial Special Issue on Artificial Neural Networks and Statistical Pattern Recognition , 1997, IEEE Trans. Neural Networks.

[28]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[29]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[30]  Martin G. Bello,et al.  Enhanced training algorithms, and integrated training/architecture selection for multilayer perceptron networks , 1992, IEEE Trans. Neural Networks.

[31]  Régis Lengellé,et al.  Training MLPs layer by layer using an objective function for internal representations , 1996, Neural Networks.

[32]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[33]  David A. Landgrebe,et al.  Feature Extraction Based on Decision Boundaries , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Emile H. L. Aarts,et al.  The construction of minimal multilayered perceptrons: A case study for sorting , 1993, Neurocomputing.

[35]  Separable Regions.  On Hidden Nodes for Neural Nets , 1989 .

[36]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[37]  Patrick Gallinari,et al.  Multilayer perceptrons and data analysis , 1988, IEEE 1988 International Conference on Neural Networks.

[38]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[39]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[40]  Masahiko Arai,et al.  Bounds on the number of hidden units in binary-valued three-layer neural networks , 1993, Neural Networks.

[41]  P. GALLINARI,et al.  On the relations between discriminant analysis and multilayer perceptrons , 1991, Neural Networks.

[42]  Kishan G. Mehrotra,et al.  Bounds on the number of samples needed for neural learning , 1991, IEEE Trans. Neural Networks.

[43]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[44]  Lars Kai Hansen,et al.  Neural classifier construction using regularization, pruning and test error estimation , 1998, Neural Networks.