Variable Projection Methods for Large—Scale Quadratic Optimization in Data Analysis Applications

This paper concerns with the numerical evaluation of the variable projection method for quadratic programming problems in three data analysis applications. The three applications give rise to large—scale quadratic programs with remarkable differences in the Hessian definition and/or in the structure of the constraints.

[1]  O. Mangasarian,et al.  Parallel gradient projection successive overrelaxation for symmetric linear complementarity problems and linear programs , 1988 .

[2]  Valeria Ruggiero,et al.  Splitting methods for constrained quadratic programs in data analysis , 1996 .

[3]  Valeria Ruggiero,et al.  Splitting methods for quadratic optimization in data analysis , 1997, Int. J. Comput. Math..

[4]  Luca Zanni,et al.  A parallel solver for large quadratic programs in training support vector machines , 2003, Parallel Comput..

[5]  Panos M. Pardalos,et al.  An algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds , 1990, Math. Program..

[6]  T. Zanni.  Large quadratic programs in training Gaussian support vector machines , 2005 .

[7]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[8]  R. Vanderbei.  LOQO:an interior point code for quadratic programming , 1999 .

[9]  Nira Dyn,et al.  The numerical solution of equality constrained quadratic programming problems , 1983 .

[10]  L. Zanni,et al.  Variable projection methods for large convex quadratic programs , 2000 .

[11]  Massimiliano Pontil,et al.  Properties of Support Vector Machines , 1998, Neural Computation.

[13]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[14]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[15]  C. Cryer.  The Solution of a Quadratic Programming Problem Using Systematic Overrelaxation , 1971 .

[16]  F. Giannessi,et al.  Nonlinear Optimization and Related Topics , 2000 .

[17]  C. Durazzi,et al.  A Random Generator for Large-Scale Linearly Constrained Quadratic Programming Test Problems , 2000 .

[18]  L. Zanni,et al.  Projection-type methods for large convex quadratic programs: theory and computational experience , 2000 .

[19]  Federico Girosi,et al.  Support Vector Machines: Training and Applications , 1997 .

[20]  J. Gregory,et al.  Compatable smooth interpolation in triangles , 1975 .

[21]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[22]  P. Pardalos,et al.  Equilibrium problems : nonsmooth optimization and variational inequality models , 2004 .

[23]  I. M. Klucewicz.  A piecewise c1 interpolant to arbitrarily spaced data , 1978 .

[24]  Emanuele Galligani,et al.  C1 surface interpolation with constraints , 1993, Numerical Algorithms.

[25]  W. J. Gordon,et al.  Smooth interpolation in triangles , 1973 .

[26]  P. Tseng,et al.  Modified Projection-Type Methods for Monotone Variational Inequalities , 1996 .

[27]  Emanuele Galligani,et al.  On a quadratic functional occurring in a bivariate scattered data interpolation problem , 2002, ANNALI DELL UNIVERSITA DI FERRARA.

[28]  A. Verri,et al.  ANALISI NUMERICA: METODI E SOFTWARE MATEMATICO , 1998 .

[29]  Gabor T. Herman,et al.  Image Reconstruction From Projections , 1975, Real Time Imaging.

[30]  O. Mangasarian,et al.  Parallel successive overrelaxation methods for symmetric linear complementarity problems and linear programs , 1987 .

[31]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[32]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[33]  Charles L. Lawson,et al.  $C^1$ surface interpolation for scattered data on a sphere , 1984 .

[34]  S. Venit,et al.  Numerical Analysis: A Second Course. , 1974 .

[35]  J. Ortega.  Numerical Analysis: A Second Course , 1974 .

[36]  Giulio Casciola,et al.  Algorithm 677 C1 surface interpolation , 1989, TOMS.

[37]  Valeria Ruggiero,et al.  Splitting methods and parallel solution of constrained quadratic programs , 1997 .

[38]  L. Zanni,et al.  A Modified Projection Algorithm for Large Strictly-Convex Quadratic Programs , 2000 .

[39]  Gabor T. Herman,et al.  Image Reconstruction From Projections , 1975, Real Time Imaging.

[40]  L. Zanni,et al.  On the Efficiency of Splitting and Projection Methods for Large Strictly Convex Quadratic Programs , 2000 .

[41]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[42]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[43]  C. Lawson.  Software for C1 Surface Interpolation , 1977 .

[44]  Hiroshi Akima,et al.  A Method of Bivariate Interpolation and Smooth Surface Fitting for Irregularly Distributed Data Points , 1978, TOMS.

[45]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Michael C. Ferris,et al.  Interior-Point Methods for Massive Support Vector Machines , 2002, SIAM J. Optim..

[47]  P. Marcotte,et al.  On the convergence of projection methods: Application to the decomposition of affine variational inequalities , 1995 .

[48]  G. Nielson.  A method for interpolating scattered data based upon a minimum norm network , 1983 .

[49]  B. R. Hunt,et al.  Digital Image Restoration , 1977 .

[50]  R. Varga,et al.  Block diagonally dominant matrices and generalizations of the Gerschgorin circle theorem , 1962 .

[51]  Gregory M. Nielson,et al.  A method for construction of surfaces under tension , 1984 .

[52]  L. Zanni,et al.  Parallel Solution of Large Scale Quadratic Programs , 1998 .

[53]  L. Zanni,et al.  AN OVERVIEW ON PROJECTION-TYPE METHODS FOR CONVEX LARGE-SCALE QUADRATIC PROGRAMS * , 2001 .

[54]  J. Pang,et al.  Iterative methods for large convex quadratic programs: a survey , 1987 .

[55]  Stavros A. Zenios,et al.  Massively Parallel Algorithms for Singly Constrained Convex Programs , 1992, INFORMS J. Comput..

[56]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .