Optimization algorithms for big data with application in wireless networks

This chapter proposes the use of modern first-order large-scale optimization techniques to manage a cloudbased densely deployed next-generation wireless network. In the first part of the chapter we survey a few popular first-order methods for large-scale optimization, including the block coordinate descent (BCD) method, the block successive upper-bound minimization (BSUM) method and the alternating direction method of multipliers (ADMM). In the second part of the chapter, we show that many difficult problems in managing large wireless networks can be solved efficiently and in a parallel manner, by modern first-order optimization methods. Extensive numerical results are provided to demonstrate the benefit of the proposed approach. Disciplines Signal Processing | Systems and Communications | Systems Engineering Comments This is a chapter published as Mingyi Hong, Wei-Cheng Liao, Ruoyu Sun and Zhi-Quan Luo "Optimization Algorithms for Big Data with Application in Wireless Networks," in Big Data over Networks, ed. Shuguang Cui, Alfred O. Hero III, Zhi-quan Luo, and Jose M. F. Moura (Cambridge: Cambridge University Press, 2016), pp. 66-100. Posted with permission. This book chapter is available at Iowa State University Digital Repository: https://lib.dr.iastate.edu/imse_pubs/171 I 3 Optimization algorithms for big data with application in wireless networks Mingyi Hong, Wei-Cheng Liao, Ruoyu Sun, and Zhi-Quan Luo This chapter proposes the use of modern first-order large-scale optimization techniques to manage a cloud-based densely deployed next-generation wireless network. In the first part of the chapter we survey a few popular first-order methods for large-scale optimization, including the block coordinate descent (BCD) method, the block successive upper-bound minimization (BSUM) method and the alternating direction method of multipliers (ADMM). In the second part of the chapter, we show that many difficult problems in managing large wireless networks can be solved efficiently and in a parallel manner, by modern first-order optimization methods. Extensive numerical results are provided to demonstrate the benefit of the proposed approach.

[1]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[2]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[3]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[4]  Qiang Li,et al.  Decomposition by Successive Convex Approximation: A Unifying Approach for Linear Transceiver Design in Heterogeneous Networks , 2012, IEEE Transactions on Wireless Communications.

[5]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[6]  John M. Cioffi,et al.  Weighted Sum-Rate Maximization Using Weighted MMSE for MIMO-BC Beamforming Design , 2008, 2009 IEEE International Conference on Communications.

[7]  Mikhail V. Solodov,et al.  On the Convergence of Constrained Parallel Variable Distribution Algorithms , 1998, SIAM J. Optim..

[8]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[9]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[10]  Z.-Q. Luo,et al.  Error bounds and convergence analysis of feasible descent methods: a general approach , 1993, Ann. Oper. Res..

[11]  Ambuj Tewari,et al.  Stochastic methods for l1 regularized loss minimization , 2009, ICML '09.

[12]  Ambuj Tewari,et al.  On the Nonasymptotic Convergence of Cyclic Coordinate Descent Methods , 2013, SIAM J. Optim..

[13]  P. Tseng.  Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[14]  Zhi-Quan Luo,et al.  Linear transceiver design for a MIMO interfering broadcast channel achieving max-min fairness , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[15]  D. Bertsekas,et al.  Relaxation methods for network flow problems with convex arc costs , 1987 .

[16]  Shiqian Ma,et al.  Sparse Inverse Covariance Selection via Alternating Linearization Methods , 2010, NIPS.

[17]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[18]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[19]  Renato D. C. Monteiro,et al.  Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers , 2013, SIAM J. Optim..

[20]  Francisco Facchinei,et al.  Flexible parallel algorithms for big data optimization , 2013, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  Gordon P. Wright,et al.  Technical Note - A General Inner Approximation Algorithm for Nonconvex Mathematical Programs , 1978, Oper. Res..

[22]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[23]  Lin Xiao,et al.  On the complexity analysis of randomized block-coordinate descent methods , 2013, Mathematical Programming.

[24]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[25]  HongMingyi,et al.  Iteration complexity analysis of block coordinate descent methods , 2017 .

[26]  Richard G. Baraniuk,et al.  Fast Alternating Direction Optimization Methods , 2014, SIAM J. Imaging Sci..

[27]  Junfeng Yang,et al.  An Efficient TVL1 Algorithm for Deblurring Multichannel Images Corrupted by Impulsive Noise , 2009, SIAM J. Sci. Comput..

[28]  D. Hunter,et al.  Quantile Regression via an MM Algorithm , 2000 .

[29]  Zhi-Quan Luo,et al.  Iteration complexity analysis of block coordinate descent methods , 2013, Mathematical Programming.

[30]  P. Tseng,et al.  On the convergence of the coordinate descent method for convex differentiable minimization , 1992 .

[31]  P. Tseng,et al.  On the linear convergence of descent methods for convex essentially smooth minimization , 1992 .

[32]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[33]  Daniel Boley.  Linear Convergence of ADMM on a Model Problem , 2012 .

[34]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[35]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[36]  Bingsheng He,et al.  Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming , 2011 .

[37]  Peter Richtárik,et al.  Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function , 2011, Mathematical Programming.

[38]  Shuzhong Zhang,et al.  Maximum Block Improvement and Polynomial Optimization , 2012, SIAM J. Optim..

[39]  Jack Yurkiewicz,et al.  Constrained optimization and Lagrange multiplier methods, by D. P. Bertsekas, Academic Press, New York, 1982, 395 pp. Price: $65.00 , 1985, Networks.

[40]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[41]  Zhi-Quan Luo,et al.  Joint Base Station Clustering and Beamformer Design for Partial Coordinated Transmission in Heterogeneous Networks , 2012, IEEE Journal on Selected Areas in Communications.

[42]  Francisco Facchinei,et al.  Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systems , 2013, IEEE Transactions on Signal Processing.

[43]  Dimitri P. Bertsekas,et al.  Nonlinear Programming 2 , 2005 .

[44]  Shlomo Shamai,et al.  Joint Precoding and Multivariate Backhaul Compression for the Downlink of Cloud Radio Access Networks , 2013, IEEE Transactions on Signal Processing.

[45]  Xu Li,et al.  Min Flow Rate Maximization for Software Defined Radio Access Networks , 2013, IEEE Journal on Selected Areas in Communications.

[46]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[47]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[48]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[49]  Asuman Ozdaglar,et al.  Cooperative distributed multi-agent optimization , 2010, Convex Optimization in Signal Processing and Communications.

[50]  Lieven De Lathauwer,et al.  Swamp reducing technique for tensor decomposition , 2008, 2008 16th European Signal Processing Conference.

[51]  Jeffrey G. Andrews,et al.  Seven ways that HetNets are a cellular paradigm shift , 2013, IEEE Communications Magazine.

[52]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[53]  Shiqian Ma,et al.  Solving Multiple-Block Separable Convex Minimization Problems Using Two-Block Alternating Direction Method of Multipliers , 2013, ArXiv.

[54]  Zhi-Quan Luo,et al.  An iteratively weighted MMSE approach to distributed sum-utility maximization for a MIMO interfering broadcast channel , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).