Improving the Generalization of Neural Networks by Changing the Structure of Artificial Neuron

This paper introduces a change in the structure of an artificial neuron (McCulloch and Pitts), to improve the performance of the feed forward artificial neural networks like the multi-layer perceptron networks. Results on function approximation task and three pattern recognition problems show that the performance of a neural network can be improved by a simple change in its traditional structure. The first problem is about approximation of a complicated function and the other tasks are three pattern classification problems which we have considered the digit, face and 3D object recognition experiments for evaluation. The results of the experiments confirm the improvement of the generalization of the proposed method in compared to the traditional neural network structure.

[1]  Nikola Kasabov,et al.  Foundations Of Neural Networks, Fuzzy Systems, And Knowledge Engineering [Books in Brief] , 1996, IEEE Transactions on Neural Networks.

[2]  C. Lee Giles,et al.  What Size Neural Network Gives Optimal Generalization? Convergence Properties of Backpropagation , 1998 .

[3]  Cheng-Wu Chen,et al.  Improving the generalization performance of RBF neural networks using a linear regression technique , 2009, Expert Syst. Appl..

[4]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[5]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[6]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[7]  Brian V. Funt,et al.  Is Machine Colour Constancy Good Enough? , 1998, ECCV.

[8]  Ricardo H. C. Takahashi,et al.  Improving generalization of MLPs with multi-objective optimization , 2000, Neurocomputing.

[9]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[10]  F. Siraj,et al.  Improving Generalization of Neural Networks Using MLP Discriminant Based on Multiple Classifiers Failures , 2010, 2010 Second International Conference on Computational Intelligence, Modelling and Simulation.

[11]  B. R. Menezes,et al.  Improving generalization of MLPs with sliding mode control and the Levenberg-Marquardt algorithm , 2007, Neurocomputing.

[12]  Shingo Mabu,et al.  Enhancing the generalization ability of neural networks through controlling the hidden layers , 2009, Appl. Soft Comput..

[13]  Juan R. Rabuñal,et al.  Artificial Neural Networks in Real-Life Applications , 2005 .

[14]  Lutz Prechelt,et al.  Automatic early stopping using cross validation: quantifying the criteria , 1998, Neural Networks.

[15]  M. Stone.  Asymptotics for and against cross-validation , 1977 .

[16]  Vladimir Cherkassky,et al.  Factors controlling generalization ability of MLP networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[17]  Eduardo Ros,et al.  A New Measurement of Noise Immunity and Generalization Ability for MLPs , 1999, Int. J. Neural Syst..

[18]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.