Maximizing margins of multilayer neural networks

According to the CARVE algorithm, any pattern classification problem can be synthesized in three layers without misclassification. In this paper, we propose to train multilayer neural network classifiers based on the CARVE algorithm. In hidden layer training, we find a hyperplane that separates a set of data belonging to one class from the remaining data. Then, we remove the separated data from the training data, and repeat this procedure until only the data belonging to one class remain. In determining the hyperplane, we maximize margins heuristically so that data of one class are on one side of the hyperplane. In output layer training, we determine the hyperplane by a quadratic optimization technique. The performance of this new algorithm is evaluated by some benchmark data sets.

[1]  Steven Young,et al.  CARVE — A Constructive Algorithm for Real Valued Examples , 1994 .

[2]  S. Abe,et al.  Fuzzy support vector machines for pattern classification , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[3]  Shigeo Abe,et al.  Input Layer Optimization of Neural Networks by Sensitivity Analysis and its Application to Recognition of Numerals , 1991 .

[4]  阿部 重夫 Pattern classification : neuro-fuzzy methods and their comparison , 2001 .

[5]  D. Roobaert DirectSVM: a fast and simple support vector machine perceptron , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).