A Computational Geometry Approach for Pareto-Optimal Selection of Neural Networks

This paper presents a Pareto-optimal selection strategy for multiobjective learning that is based on the geometry of the separation margin between classes. The Gabriel Graph, a method borrowed from Computational Geometry, is constructed in order to obtain margin patterns and class borders. From border edges, a target separator is obtained in order to obtain a large margin classifier. The selected model from the generated Pareto-set is the one that is closer to the target separator. The method presents robustness in both synthetic and real benchmark datasets. It is efficient for Pareto-Optimal selection of neural networks and no claim is made that the obtained solution is equivalent to a maximum margin separator.

[1]  Ricardo H. C. Takahashi,et al.  Improving generalization of MLPs with multi-objective optimization , 2000, Neurocomputing.

[2]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[3]  Bernhard Sendhoff,et al.  Corrections to “Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies” [May 08 397-415] , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Robert B. Fisher,et al.  Incremental One-Class Learning with Bounded Computational Complexity , 2007, ICANN.

[5]  Ricardo H. C. Takahashi,et al.  The Usage of Golden Section in Calculating the Efficient Solution in Artificial Neural Networks Training by Multi-objective Optimization , 2007, ICANN.

[6]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[7]  Antônio de Pádua Braga,et al.  An efficient multi-objective learning algorithm for RBF neural network , 2010, Neurocomputing.

[8]  Filiberto Pla,et al.  Prototype selection for the nearest neighbour rule through proximity graphs , 1997, Pattern Recognit. Lett..

[9]  Johan A. K. Suykens,et al.  Benchmarking Least Squares Support Vector Machine Classifiers , 2004, Machine Learning.

[10]  Hirotaka Nakayama,et al.  Theory of Multiobjective Optimization , 1985 .

[11]  Bernhard Sendhoff,et al.  Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[13]  B. R. Menezes,et al.  Improving generalization of MLPs with sliding mode control and the Levenberg-Marquardt algorithm , 2007, Neurocomputing.

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  Peter L. Bartlett,et al.  For Valid Generalization the Size of the Weights is More Important than the Size of the Network , 1996, NIPS.

[16]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .