A multi-objective approach to RBF network learning

The problem of inductive supervised learning is discussed in this paper within the context of multi-objective (MOBJ) optimization. The smoothness-based apparent (effective) complexity measure for RBF networks is considered. For the specific case of RBF network, bounds on the complexity measure are formally described. As the synthetic and real-world data experiments show, the proposed MOBJ learning method is capable of efficient generalization control along with network size reduction.

[1]  Bernhard Sendhoff,et al.  Neural network regularization and ensembling using multi-objective evolutionary algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[2]  Antônio de Pádua Braga,et al.  Optimization of Neural Networks with Multi-Objective LASSO Algorithm , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[3]  Shun-ichi Amari,et al.  A Theory of Pattern Recognition , 1968 .

[4]  Peter L. Bartlett,et al.  For Valid Generalization the Size of the Weights is More Important than the Size of the Network , 1996, NIPS.

[5]  Ricardo H. C. Takahashi,et al.  Improving generalization of MLPs with multi-objective optimization , 2000, Neurocomputing.

[6]  Antônio de Pádua Braga,et al.  Training neural networks with a multi-objective sliding mode control algorithm , 2003, Neurocomputing.

[7]  Lutz Prechelt,et al.  PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms , 1994 .

[8]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[9]  Ricardo H. C. Takahashi,et al.  Multi-Objective Algorithms for Neural Networks Learning , 2006, Multi-Objective Machine Learning.

[10]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[11]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[12]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[13]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[14]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[15]  Guo-Ping Liu,et al.  Multiobjective Optimisation And Control , 2008 .

[16]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[17]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[18]  H. L. Le Roy,et al.  Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; Vol. IV , 1969 .

[19]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[20]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .