Block Alternating Optimization for Non-convex Min-max Problems: Algorithms and Applications in Signal Processing and Communications

The min-max problem, also known as the saddle point problem, can be used to formulate a wide range of applications in signal processing and wireless communications. However, existing optimization theory and methods, which mostly deal with problems with certain convex-concave structure, are not applicable for the aforementioned applications, which oftentimes involve non-convexity. In this work, we consider a general block-wise one-sided non-convex min-max problem, in which the minimization problem consists of multiple blocks and is non-convex, while the maximization problem is (strongly) concave. We propose two simple algorithms, which alternatingly perform one gradient descent-type step for each minimization block and one gradient ascent-type step for the maximization problem. For the first time, we show that such simple alternating min-max algorithms converge to first-order stationary solutions. We conduct numerical tests on a robust learning problem, and a wireless communication problem in the presence of jammers, to validate the efficiency of the proposed algorithms.

[1]  Zhi-Quan Luo,et al.  An iteratively weighted MMSE approach to distributed sum-utility maximization for a MIMO interfering broadcast channel , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Yongxin Chen,et al.  Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications , 2019, IEEE Transactions on Signal Processing.

[3]  Yunmei Chen,et al.  Optimal Primal-Dual Methods for a Class of Saddle Point Problems , 2013, SIAM J. Optim..

[4]  Michael L. Honig,et al.  Monotonic convergence of distributed interference pricing in wireless networks , 2009, 2009 IEEE International Symposium on Information Theory.

[5]  Jason D. Lee,et al.  On the Convergence and Robustness of Training GANs with Regularized Optimal Transport , 2018, NeurIPS.

[6]  Gonzalo Mateos,et al.  Distributed Sparse Linear Regression , 2010, IEEE Transactions on Signal Processing.

[7]  L. Hien,et al.  An Inexact Primal-Dual Smoothing Framework for Large-Scale Non-Bilinear Saddle Point Problems , 2017, J. Optim. Theory Appl..

[8]  Wolfgang Utschick,et al.  Comparison of Distributed Beamforming Algorithms for MIMO Interference Networks , 2013, IEEE Transactions on Signal Processing.

[9]  Rong Jin,et al.  Robust Optimization over Multiple Domains , 2018, AAAI.

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Mingrui Liu,et al.  Non-Convex Min-Max Optimization: Provable Algorithms and Applications in Machine Learning , 2018, ArXiv.

[12]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[13]  Mingyi Hong,et al.  Perturbed proximal primal–dual algorithm for nonconvex nonsmooth optimization , 2019, Math. Program..

[14]  Zhi-Quan Luo,et al.  Ieee Transactions on Signal Processing a Generalized Iterative Water-filling Algorithm for Distributed Power Control in the Presence of a Jammer , 2022 .

[15]  Zhi-Quan Luo,et al.  Semi-asynchronous routing for large scale hierarchical networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Qing Ling,et al.  Decentralized learning for wireless communications and networking , 2015, ArXiv.