Penalty dual decomposition method with application in signal processing

Many problems of recent interest in signal processing, machine learning and wireless communications can be posed as nonconvex nonsmooth optimization problems. These problems are generally difficult to solve especially when the optimization variables are nonlinearly coupled in some nonconvex constraints. In this paper, we propose an algorithm named “penalty dual decomposition” (PDD) method, for the minimization of a nonconvex nonsmooth objective subject to nonconvex constraints. We show that the PDD converges to KKT solutions under certain constraint qualification condition. Simulations corroborate the excellent performance of the PDD method.

[1]  Xiqi Gao,et al.  Joint Source-Relay Design for Full-Duplex MIMO AF Relay Systems , 2016, IEEE Transactions on Signal Processing.

[2]  Yong Zhang,et al.  Sparse Approximation via Penalty Decomposition Methods , 2012, SIAM J. Optim..

[3]  Xiangfeng Wang,et al.  Nonnegative matrix factorization using ADMM: Algorithm and convergence analysis , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  E. Balder.  On generalized gradients and optimization ∗ , 2008 .

[5]  Yue Rong,et al.  Joint Source and Relay Optimization for Two-Way Linear Non-Regenerative MIMO Relay Communications , 2012, IEEE Transactions on Signal Processing.

[6]  Nikos D. Sidiropoulos,et al.  Quality of Service and Max-Min Fair Transmit Beamforming to Multiple Cochannel Multicast Groups , 2008, IEEE Transactions on Signal Processing.

[7]  Alexey F. Izmailov,et al.  Optimality Conditions for Irregular Inequality-Constrained Problems , 2001, SIAM J. Control. Optim..

[8]  Alexey F. Izmailov,et al.  On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers , 2011, Math. Program..

[9]  Zhaosong Lu,et al.  Penalty decomposition methods for rank minimization , 2010, Optim. Methods Softw..

[10]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[11]  Stéphane Canu,et al.  Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming , 2009, IEEE Transactions on Signal Processing.

[12]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[13]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[14]  Haesun Park,et al.  SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering , 2014, Journal of Global Optimization.

[15]  Björn E. Ottersten,et al.  Grab-n-Pull: An optimization framework for fairness-achieving networks , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Benjamin Friedlander,et al.  Bilinear compressed sensing for array self-calibration , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[17]  Ying-Chang Liang,et al.  Joint Beamforming and Power Control for Multiantenna Relay Broadcast Channel With QoS Constraints , 2009, IEEE Transactions on Signal Processing.

[18]  Zhi-Quan Luo,et al.  SINR Constrained Beamforming for a MIMO Multi-User Downlink System: Algorithms and Convergence Analysis , 2012, IEEE Transactions on Signal Processing.

[19]  Michael P. Friedlander,et al.  A Globally Convergent Linearly Constrained Lagrangian Method for Nonlinear Optimization , 2005, SIAM J. Optim..

[20]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[21]  Jiang Wang,et al.  A Globally Optimal Bilinear Programming Approach to the Design of Approximate Hilbert Pairs of Orthonormal Wavelet Bases , 2010, IEEE Transactions on Signal Processing.

[22]  Philippe J. Sartori,et al.  Cooperative Algorithms for MIMO Amplify-and-Forward Relay Networks , 2011, IEEE Transactions on Signal Processing.

[23]  Cédric Févotte,et al.  Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Nikos D. Sidiropoulos,et al.  Blind Separation of Quasi-Stationary Sources: Exploiting Convex Geometry in Covariance Domain , 2015, IEEE Transactions on Signal Processing.

[25]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[26]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[27]  Mikhail V. Solodov,et al.  Local Convergence of Exact and Inexact Augmented Lagrangian Methods under the Second-Order Sufficient Optimality Condition , 2012, SIAM J. Optim..

[28]  Wei Yu,et al.  Sparse Beamforming and User-Centric Clustering for Downlink Cloud Radio Access Network , 2014, IEEE Access.

[29]  Wim Michiels,et al.  Combining Convex–Concave Decompositions and Linearization Approaches for Solving BMIs, With Application to Static Output Feedback , 2011, IEEE Transactions on Automatic Control.