First-Order Methods for Fast Feasibility Pursuit of Non-convex QCQPs

Quadratically constrained quadratic programming (QCQP) is NP-hard in its general non-convex form, yet it frequently arises in various engineering applications. Several polynomial-time approximation algorithms exist for non-convex QCQP problems (QCQPs), but their success hinges upon the ability to find at least one feasible point—which is also hard for a general problem instance. In this paper, we present a heuristic framework for computing feasible points of general non-convex QCQPs using simple first-order methods. Our approach features low computational and memory requirements, which makes it well suited for application to large-scale problems. While a priori it may appear that these benefits come at the expense of technical sophistication, rendering our approach too simple to even merit consideration for a non-convex and NP-hard problem, we provide compelling empirical evidence to the contrary. Experiments on synthetic as well as real-world instances of non-convex QCQPs reveal the surprising effectiveness of first-order methods compared to more established and sophisticated alternatives.

[1]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[2]  Saeed Ghadimi,et al.  Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization , 2013, Mathematical Programming.

[3]  Michael I. Jordan,et al.  Gradient Descent Only Converges to Minimizers , 2016, COLT.

[4]  Saeed Ghadimi,et al.  Accelerated gradient methods for nonconvex nonlinear and stochastic programming , 2013, Mathematical Programming.

[5]  Nikos D. Sidiropoulos,et al.  Fast Approximation Algorithms for a Class of Non-convex QCQP Problems Using First-Order Methods , 2017, IEEE Transactions on Signal Processing.

[6]  Gang Wang,et al.  Power system state estimation via feasible point pursuit , 2017, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[7]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[8]  Francisco Facchinei,et al.  Parallel and Distributed Methods for Nonconvex Optimization-Part I: Theory , 2014 .

[9]  Georgios B. Giannakis,et al.  Monitoring and Optimization for Power Grids: A Signal Processing Perspective , 2013, IEEE Signal Processing Magazine.

[10]  Nikos D. Sidiropoulos,et al.  Fast feasibility pursuit for non-convex QCQPS via first-order methods , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[12]  Max Simchowitz,et al.  Low-rank Solutions of Linear Matrix Equations via Procrustes Flow , 2015, ICML.

[13]  Meisam Razaviyayn,et al.  Successive Convex Approximation: Analysis and Applications , 2014 .

[14]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[15]  Lin Xiao,et al.  A Proximal Stochastic Gradient Method with Progressive Variance Reduction , 2014, SIAM J. Optim..

[16]  Alexander J. Smola,et al.  Stochastic Variance Reduction for Nonconvex Optimization , 2016, ICML.

[17]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[18]  Francisco Facchinei,et al.  Parallel and Distributed Methods for Constrained Nonconvex Optimization—Part I: Theory , 2016, IEEE Transactions on Signal Processing.

[19]  Nikos D. Sidiropoulos,et al.  Max-min feasible point pursuit for non-convex QCQP , 2015, 2015 49th Asilomar Conference on Signals, Systems and Computers.

[20]  Gordon P. Wright,et al.  Technical Note - A General Inner Approximation Algorithm for Nonconvex Mathematical Programs , 1978, Oper. Res..

[21]  Martin J. Wainwright,et al.  Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees , 2015, ArXiv.

[22]  Nikos D. Sidiropoulos,et al.  Consensus-ADMM for General Quadratically Constrained Quadratic Programming , 2016, IEEE Transactions on Signal Processing.

[23]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[24]  Zhang Liu,et al.  Interior-point methods for large-scale cone programming , 2011 .

[25]  Saeed Ghadimi,et al.  Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming , 2013, SIAM J. Optim..

[26]  Pascal Van Hentenryck,et al.  AC-Feasibility on Tree Networks is NP-Hard , 2014, IEEE Transactions on Power Systems.

[27]  Zhi-Quan Luo,et al.  A Stochastic Successive Minimization Method for Nonsmooth Nonconvex Optimization with Applications to Transceiver Design in Wireless Communication Networks , 2013, Mathematical Programming.

[28]  Andrew Cotter,et al.  Stochastic Optimization for Machine Learning , 2013, ArXiv.

[29]  Christopher De Sa,et al.  Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems , 2014, ICML.

[30]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[31]  Y. Ye.  Data Randomness Makes Optimization Problems Easier to Solve ? , 2016 .

[32]  Martin Connors,et al.  Optimization Models , 2014 .

[33]  Amir Beck,et al.  A sequential parametric convex approximation method with applications to nonconvex truss topology design problems , 2010, J. Glob. Optim..

[34]  Nathan Srebro,et al.  Global Optimality of Local Search for Low Rank Matrix Recovery , 2016, NIPS.

[35]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[36]  Zeyuan Allen Zhu,et al.  Variance Reduction for Faster Non-Convex Optimization , 2016, ICML.

[37]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[38]  Nikos D. Sidiropoulos,et al.  Feasible Point Pursuit and Successive Approximation of Non-Convex QCQPs , 2014, IEEE Signal Processing Letters.

[39]  Alexander J. Smola,et al.  Fast Stochastic Methods for Nonsmooth Nonconvex Optimization , 2016, ArXiv.

[40]  Zhi-Quan Luo,et al.  Semidefinite Relaxation of Quadratic Optimization Problems , 2010, IEEE Signal Processing Magazine.

[41]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[42]  Tong Zhang,et al.  Accelerating Stochastic Gradient Descent using Predictive Variance Reduction , 2013, NIPS.

[43]  Carleton Coffrin,et al.  NESTA, The NICTA Energy System Test Case Archive , 2014, ArXiv.

[44]  Fred C. Schweppe,et al.  Power System Static-State Estimation, Part I: Exact Model , 1970 .

[45]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[46]  Anastasios Kyrillidis,et al.  Dropping Convexity for Faster Semi-definite Optimization , 2015, COLT.

[47]  Zhi-Quan Luo,et al.  Guaranteed Matrix Completion via Non-Convex Factorization , 2014, IEEE Transactions on Information Theory.

[48]  Zhi-Quan Luo,et al.  SDP relaxation of homogeneous quadratic optimization: Approximation bounds and applications , 2009 .

[49]  R D Zimmerman,et al.  MATPOWER: Steady-State Operations, Planning, and Analysis Tools for Power Systems Research and Education , 2011, IEEE Transactions on Power Systems.