Adaptive constraint reduction for convex quadratic programming

We propose an adaptive, constraint-reduced, primal-dual interior-point algorithm for convex quadratic programming with many more inequality constraints than variables. We reduce the computational effort by assembling, instead of the exact normal-equation matrix, an approximate matrix from a well chosen index set which includes indices of constraints that seem to be most critical. Starting with a large portion of the constraints, our proposed scheme excludes more unnecessary constraints at later iterations. We provide proofs for the global convergence and the quadratic local convergence rate of an affine-scaling variant. Numerical experiments on random problems, on a data-fitting problem, and on a problem in array pattern synthesis show the effectiveness of the constraint reduction in decreasing the time per iteration without significantly affecting the number of iterations. We note that a similar constraint-reduction approach can be applied to algorithms of Mehrotra’s predictor-corrector type, although no convergence theory is supplied.

[1]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[2]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[3]  Yinyu Ye,et al.  An O(n3L) potential reduction algorithm for linear programming , 1991, Math. Program..

[4]  Jian L. Zhou,et al.  A Simple, Quadratically Convergent Interior Point Algorithm for Linear Programming and Convex Quadratic Programming , 1994 .

[5]  J. H. Wilkinson,et al.  Reliable Numerical Computation. , 1992 .

[6]  M. Chial,et al.  in simple , 2003 .

[7]  Alston S. Householder,et al.  The Theory of Matrices in Numerical Analysis , 1964 .

[8]  Dianne P. O'Leary,et al.  Adaptive use of iterative methods in predictor–corrector interior point methods for linear programming , 2000, Numerical Algorithms.

[9]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[10]  Yinyu Ye,et al.  A “build-down” scheme for linear programming , 1990, Math. Program..

[11]  G. A. Watson,et al.  Choice of norms for data fitting and function approximation , 1998, Acta Numerica.

[12]  Kaoru Tone,et al.  An active-set strategy in an interior point method for linear programming , 1991, Math. Program..

[13]  W. Hager,et al.  Large Scale Optimization : State of the Art , 1993 .

[14]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[15]  Ingvar Claesson,et al.  A semi-infinite quadratic programming algorithm with applications to array pattern synthesis , 2001 .

[16]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[17]  Dick den Hertog,et al.  A build-up variant of the path-following method for LP , 1991 .

[18]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[19]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[20]  N. Higham.  Analysis of the Cholesky Decomposition of a Semi-definite Matrix , 1990 .

[21]  Yinyu Ye,et al.  A Potential Reduction Algorithm Allowing Column Generation , 1992, SIAM J. Optim..

[22]  Jin Hyuk Jung,et al.  Adaptive Constraint Reduction for Convex Quadratic Programming and Training Support Vector Machines , 2008 .

[23]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[24]  Tamás Terlaky,et al.  Adding and Deleting Constraints in the Logarithmic Barrier Method for LP , 1994 .

[25]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[26]  Dianne P. O'Leary,et al.  Adaptive constraint reduction for training support vector machines. , 2008 .

[27]  Dianne P. O'Leary,et al.  A constraint-reduced variant of Mehrotra’s predictor-corrector algorithm , 2012, Comput. Optim. Appl..

[28]  Jie Sun,et al.  An Analytic Center Based Column Generation Algorithm for Convex Quadratic Feasibility Problems , 1998, SIAM J. Optim..

[29]  Yin Zhang,et al.  Solving large-scale linear programs by interior-point methods under the Matlab ∗ Environment † , 1998 .

[30]  Michael C. Ferris,et al.  Interior-Point Methods for Massive Support Vector Machines , 2002, SIAM J. Optim..

[31]  Pierre-Antoine Absil,et al.  Constraint Reduction for Linear Programs with Many Inequality Constraints , 2006, SIAM J. Optim..

[32]  Pierre-Antoine Absil,et al.  Newton-KKT interior-point methods for indefinite quadratic programming , 2007, Comput. Optim. Appl..

[33]  J. Lockard,et al.  University of Maryland , 1844, The American journal of dental science.

[34]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[35]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[36]  G. Dantzig,et al.  A Build-Up Interior Method for Linear Programming: Affine Scaling Form , 1990 .

[37]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[38]  S. Nash,et al.  Linear and Nonlinear Optimization , 2008 .