Simple Recurrent Neural Networks for Support Vector Machine Training

We show how to implement a simple procedure for support vector machine training as a recurrent neural network. Invoking the fact that support vector machines can be trained using Frank-Wolfe optimization which in turn can be seen as a form of reservoir computing, we obtain a model that is of simpler structure and can be implemented more easily than those proposed in previous contributions.

[1]  Kenneth L. Clarkson,et al.  Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm , 2008, SODA '08.

[2]  Davide Anguita,et al.  Improved neural network for SVM learning , 2002, IEEE Trans. Neural Networks.

[3]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[4]  Pascal Vincent,et al.  A Neural Support Vector Network architecture with adaptive kernels , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[5]  Elisa Ricci,et al.  Analog neural network for support vector machine learning , 2006, IEEE Transactions on Neural Networks.

[6]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[7]  Shigeo Abe,et al.  Comparison of L1 and L2 support vector machines , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[8]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[9]  Rafet Sifa.  An Overview of Frank-Wolfe Optimization for Stochasticity Constrained Interpretable Matrix and Tensor Factorization , 2018, ICANN.

[10]  Xiaolin Hu,et al.  A compact neural network for training support vector machines , 2012, Neurocomputing.

[11]  Martin Jaggi,et al.  Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization , 2013, ICML.

[12]  Wlodzislaw Duch,et al.  Support Vector Neural Training , 2005, ICANN.

[13]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[14]  Magnus Jändel,et al.  Biologically relevant neural network architectures for support vector machines , 2014, Neural Networks.

[15]  Christian Bauckhage.  A Neural Network Implementation of Frank-Wolfe Optimization , 2017, ICANN.

[16]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.