Geometrical synthesis of MLP neural networks

This paper presents a new constructive algorithm to design multilayer perceptron networks used as classifiers. The resulting networks are able to classify patterns defined in a real domain. The proposed procedure allows us to automatically determine both the number of neurons and the synaptic weights of networks with a single hidden layer. The approach is based on linear programming. It avoids the typical local minima problems of error back propagation and assures convergence of the method. Furthermore, it will be shown in this paper that the presented procedure leads to single-hidden layer neural networks able to solve any problem in classifying a finite number of patterns. The performances of the proposed algorithm have been tested on some benchmark problems, and they have been compared with those of different approaches.

[1]  Lutz Prechelt,et al.  PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms , 1994 .

[2]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[3]  Pedro E. López-de-Teruel,et al.  Nonlinear kernel-based statistical pattern analysis , 2001, IEEE Trans. Neural Networks.

[4]  Alfred Jean Philippe Lauret,et al.  A node pruning algorithm based on a Fourier amplitude sensitivity test method , 2006, IEEE Transactions on Neural Networks.

[5]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[6]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[7]  Sung-Kwon Park,et al.  The geometrical learning of binary neural networks , 1995, IEEE Trans. Neural Networks.

[8]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[9]  Zoran Obradovic,et al.  Constructively learning a near-minimal neural network architecture , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[10]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[11]  Dimitris Taptsinos.  Engineering problems - neural network solutions: Proceedings of the International Conference on Engineering Applications of Neural Networks (EANN2000) 17-19 July 2000, Kingston University, England , 2000 .

[12]  Sungkwon Park,et al.  A training algorithm for discrete multilayer perceptrons , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[13]  Jesús Cid-Sueiro,et al.  A fixed-point algorithm to minimax learning with neural networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[14]  Carlos Cabrelli,et al.  A constructive algorithm to solve "convex recursive deletion" (CoRD) classification problems via two-layer perceptron networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[15]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[16]  Fabio Massimo Frattale Mascioli,et al.  A constructive algorithm for binary neural networks: the oil-spot algorithm , 1995, IEEE Trans. Neural Networks.

[17]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[18]  L. Bobrowski,et al.  DIPOLAR DESIGNING OF NEURAL LAYERS FOR IMAGE SEGMENTATION , 2005 .

[19]  Panos J. Antsaklis,et al.  A simple method to derive bounds on the size and to train multilayer neural networks , 1991, IEEE Trans. Neural Networks.

[20]  Y. Takahashi.  A Mathematical Solution to a Network Designing Problem , 1996, J. Comput. Biol..

[21]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .