A continuous-time recurrent neural network for real-time support vector regression

This paper presents a continuous-time recurrent neural network described by differential equations for realtime support vector regression (SVR). The SVR is first formulated as a convex quadratic programming problem, and then a continuous-time recurrent neural network with one-layer structure is designed for training the support vector machine. Furthermore, simulation results on an illustrative example are given to demonstrate the effectiveness and performance of the proposed neural network.

[1]  Jun Wang,et al.  Neural network realization of support vector methods for pattern classification , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[2]  A. F. Filippov.  Equations with the Right-Hand Side Continuous in x and Discontinuous in t , 1988 .

[3]  Jinde Cao,et al.  A Recurrent Neural Network Based on Projection Operator for Extended General Variational Inequalities , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[5]  Aleksej F. Filippov,et al.  Differential Equations with Discontinuous Righthand Sides , 1988, Mathematics and Its Applications.

[6]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[7]  Qingshan Liu,et al.  A Recurrent Neural Network for Non-smooth Convex Programming Subject to Linear Equality and Bound Constraints , 2006, ICONIP.

[8]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[9]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[10]  Jinde Cao,et al.  A Novel Recurrent Neural Network With One Neuron and Finite-Time Convergence for $k$-Winners-Take-All Operation , 2010, IEEE Transactions on Neural Networks.

[11]  Xiaolin Hu,et al.  An Alternative Recurrent Neural Network for Solving Variational Inequalities and Related Optimization Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network with a Discontinuous Activation Function for Linear Programming , 2008, Neural Computation.

[13]  J Wang,et al.  Analysis and design of primal-dual assignment networks , 1998, IEEE Trans. Neural Networks.

[14]  Jun Wang,et al.  A one-layer recurrent neural network for support vector machine learning , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[16]  Ieee Staff.  2014 IEEE Symposium on Computational Intelligence in Control and Automation (CICA) , 2014 .

[17]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[18]  Qingshan Liu,et al.  Two k-winners-take-all networks with discontinuous activation functions , 2008, Neural Networks.

[19]  Jun Wang.  Analysis and design of a recurrent neural network for linear programming , 1993 .

[20]  Paul Dierckx,et al.  Curve and surface fitting with splines , 1994, Monographs on numerical analysis.

[21]  Olvi L. Mangasarian,et al.  A finite newton method for classification , 2002, Optim. Methods Softw..

[22]  Abdesselam Bouzerdoum,et al.  Neural network for quadratic optimization with bound constraints , 1993, IEEE Trans. Neural Networks.

[23]  Michael A. Shanblatt,et al.  Linear and quadratic programming neural network analysis , 1992, IEEE Trans. Neural Networks.

[24]  Jun Wang,et al.  A One-Layer Recurrent Neural Network for Constrained Nonsmooth Optimization , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Jun Wang,et al.  A deterministic annealing neural network for convex programming , 1994, Neural Networks.

[26]  Xiaolin Hu,et al.  A Recurrent Neural Network for Solving a Class of General Variational Inequalities , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  S. Sathiya Keerthi,et al.  A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs , 2005, J. Mach. Learn. Res..

[28]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[29]  Davide Anguita,et al.  Improved neural network for SVM learning , 2002, IEEE Trans. Neural Networks.