论文信息 - Sparse approximation using least squares support vector machines

Sparse approximation using least squares support vector machines

In least squares support vector machines (LS-SVMs) for function estimation Vapnik's /spl epsiv/-insensitive loss function has been replaced by a cost function which corresponds to a form of ridge regression. In this way nonlinear function estimation is done by solving a linear set of equations instead of solving a quadratic programming problem. The LS-SVM formulation also involves less tuning parameters. However, a drawback is that sparseness is lost in the LS-SVM case. In this paper we investigate imposing sparseness by pruning support values from the sorted support value spectrum which results from the solution to the linear system.

[1] Johan A. K. Suykens,et al. Least squares support vector machine classifiers: a large scale algorithm , 1999 .

[2] Heekuck Oh,et al. Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[3] V. Vapnik. The Support Vector Method of Function Estimation , 1998 .

[4] Bernhard Schölkopf,et al. The connection between regularization operators and support vector kernels , 1998, Neural Networks.

[5] Yann LeCun,et al. Optimal Brain Damage , 1989, NIPS.

[6] F. Girosi,et al. Networks for approximation and learning , 1990, Proc. IEEE.

[7] Johan A. K. Suykens,et al. Multiclass least squares support vector machines , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[8] Alexander Gammerman,et al. Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[9] Federico Girosi,et al. An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[10] Vladimir N. Vapnik,et al. The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[11] Vladimir Vapnik,et al. Statistical learning theory , 1998 .

[13] R. Fletcher. Practical Methods of Optimization , 1988 .

[14] Babak Hassibi,et al. Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.