Control of generalization with a bi-objective sliding mode control algorithm

This paper presents a new sliding mode control algorithm that is able to guide the trajectory of a multilayer perceptron within the plane formed by the two objectives: training set error and norm of the weight vectors. The results show that the neural networks obtained are able to generate the Pareto set, from which a model with the smallest validation error is selected.

[1]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[2]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[3]  A. Hadjidimos.  Successive overrelaxation (SOR) and related methods , 2000 .

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[6]  C. Hwang,et al.  Fuzzy Multiple Objective Decision Making: Methods And Applications , 1996 .

[7]  C. Hwang.  Multiple Objective Decision Making - Methods and Applications: A State-of-the-Art Survey , 1979 .

[8]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[9]  Antônio de Pádua Braga,et al.  Neural Networks Learning with Sliding Mode Control: The Sliding Mode Backpropagation Algorithm , 1999, Int. J. Neural Syst..

[10]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[11]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[12]  Antônio de Pádua Braga,et al.  Sliding mode algorithm for training multilayer artificial neural networks , 1998 .

[13]  Peter L. Bartlett,et al.  For Valid Generalization the Size of the Weights is More Important than the Size of the Network , 1996, NIPS.

[14]  Ricardo H. C. Takahashi,et al.  Improving generalization of MLPs with multi-objective optimization , 2000, Neurocomputing.

[15]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[16]  Vadim I. Utkin,et al.  Sliding Modes and their Application in Variable Structure Systems , 1978 .