Consensus-ADMM for General Quadratically Constrained Quadratic Programming

Nonconvex quadratically constrained quadratic programming (QCQP) problems have numerous applications in signal processing, machine learning, and wireless communications, albeit the general QCQP is NP-hard, and several interesting special cases are NP-hard as well. This paper proposes a new algorithm for general QCQP. The problem is first reformulated in consensus optimization form, to which the alternating direction method of multipliers can be applied. The reformulation is done in such a way that each of the subproblems is a QCQP with only one constraint (QCQP-1), which is efficiently solvable irrespective of (non)convexity. The core components are carefully designed to make the overall algorithm more scalable, including efficient methods for solving QCQP-1, memory efficient implementation, parallel/distributed implementation, and smart initialization. The proposed algorithm is then tested in two applications: multicast beamforming and phase retrieval. The results indicate superior performance over prior state-of-the-art methods.

[1]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[2]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[3]  M. Hayes.  The reconstruction of a multidimensional sequence from the phase or magnitude of its Fourier transform , 1982 .

[4]  Yonina C. Eldar,et al.  On convexity and identifiability in 1-D Fourier phase retrieval , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[6]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[7]  Nikos D. Sidiropoulos,et al.  High Performance Adaptive Algorithms for Single-Group Multicast Beamforming , 2015, IEEE Transactions on Signal Processing.

[8]  J R Fienup,et al.  Reconstruction of an object from the modulus of its Fourier transform. , 1978, Optics letters.

[9]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[10]  Nikos D. Sidiropoulos,et al.  Parallel Algorithms for Constrained Tensor Factorization via Alternating Direction Method of Multipliers , 2014, IEEE Transactions on Signal Processing.

[11]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[12]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[13]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Daniel Pérez Palomar,et al.  Rank-Constrained Separable Semidefinite Programming With Applications to Optimal Beamforming , 2010, IEEE Transactions on Signal Processing.

[15]  Nikos D. Sidiropoulos,et al.  Spectrum Sharing in Wireless Networks via QoS-Aware Secondary Multicast Beamforming , 2009, IEEE Transactions on Signal Processing.

[16]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[17]  Nikos D. Sidiropoulos,et al.  Phase Retrieval Using Feasible Point Pursuit: Algorithms and Cramér–Rao Bound , 2015, IEEE Transactions on Signal Processing.

[18]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[19]  Shiqian Ma,et al.  Alternating direction method of multipliers for real and complex polynomial optimization models , 2014 .

[20]  Nikos D. Sidiropoulos,et al.  Feasible Point Pursuit and Successive Approximation of Non-Convex QCQPs , 2014, IEEE Signal Processing Letters.

[21]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[22]  Nikos D. Sidiropoulos,et al.  Convex Optimization-Based Beamforming , 2010, IEEE Signal Processing Magazine.

[23]  Shuzhong Zhang,et al.  New Results on Quadratic Minimization , 2003, SIAM J. Optim..

[24]  Nikos D. Sidiropoulos,et al.  Hidden Convexity in QCQP with Toeplitz-Hermitian Quadratics , 2015, IEEE Signal Processing Letters.

[25]  Shuzhong Zhang,et al.  New results on Hermitian matrix rank-one decomposition , 2011, Math. Program..

[26]  Yonina C. Eldar,et al.  Strong Duality in Nonconvex Quadratic Optimization with Two Quadratic Constraints , 2006, SIAM J. Optim..

[27]  Nikos D. Sidiropoulos,et al.  Transmit beamforming for physical-layer multicasting , 2006, IEEE Transactions on Signal Processing.

[28]  Zhi-Quan Luo,et al.  Semidefinite Relaxation of Quadratic Optimization Problems , 2010, IEEE Signal Processing Magazine.

[29]  Yonina C. Eldar,et al.  Phase Retrieval with Application to Optical Imaging: A contemporary overview , 2015, IEEE Signal Processing Magazine.

[30]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[31]  Paul Tseng,et al.  Approximation Bounds for Quadratic Optimization with Homogeneous Quadratic Constraints , 2007, SIAM J. Optim..

[32]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[33]  Mats Bengtsson,et al.  Convex Optimization-based Beamforming : From Receive to Transmit and Network Designs , 2010 .

[34]  Markku J. Juntti,et al.  A Conic Quadratic Programming Approach to Physical Layer Multicasting for Large-Scale Antenna Arrays , 2014, IEEE Signal Processing Letters.

[35]  R. Gerchberg.  A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .