Quantized consensus ADMM for multi-agent distributed optimization

This paper considers multi-agent distributed optimization with quantized communication which is needed when inter-agent communications are subject to finite capacity and other practical constraints. To minimize the global objective formed by a sum of local convex functions, we develop a quantized distributed algorithm based on the alternating direction method of multipliers (ADMM). Under certain convexity assumptions, it is shown that the proposed algorithm converges to a consensus within log1+η Ω iterations, where η > 0 depends on the network topology and the local objectives, and O is a polynomial fraction depending on the quantization resolution, the distance between initial and optimal variable values, the local objectives, and the network topology. We also obtain a tight upper bound on the consensus error which does not depend on the size of the network.

[1]  Gonzalo Mateos,et al.  Distributed Sparse Linear Regression , 2010, IEEE Transactions on Signal Processing.

[2]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[3]  Georgios B. Giannakis,et al.  Distributed Spectrum Sensing for Cognitive Radio Networks by Exploiting Sparsity , 2010, IEEE Transactions on Signal Processing.

[4]  W. Rudin.  Principles of mathematical analysis , 1964 .

[5]  John N. Tsitsiklis,et al.  Distributed subgradient methods and quantization effects , 2008, 2008 47th IEEE Conference on Decision and Control.

[6]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[7]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[8]  Shengyuan Xu,et al.  Distributed dual averaging method for multi-agent optimization with quantized communication , 2012, Syst. Control. Lett..

[9]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[10]  Ufuk Topcu,et al.  Optimal decentralized protocol for electric vehicle charging , 2011, IEEE Transactions on Power Systems.

[11]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[12]  Gregory R. Andrews,et al.  Foundations of Multithreaded, Parallel, and Distributed Programming , 1999 .

[13]  Qing Ling,et al.  Decentralized linearized alternating direction method of multipliers , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Frank Harary,et al.  Graph Theory , 2016 .

[15]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[16]  D. Cvetkovic,et al.  Signless Laplacians of finite graphs , 2007 .

[17]  Xiangfeng Wang,et al.  Multi-Agent Distributed Optimization via Inexact Consensus ADMM , 2014, IEEE Transactions on Signal Processing.

[18]  Wei Ren,et al.  Information consensus in multivehicle cooperative control , 2007, IEEE Control Systems.

[19]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[20]  Biao Chen,et al.  Quantized Consensus by the ADMM: Probabilistic Versus Deterministic Quantizers , 2015, IEEE Transactions on Signal Processing.

[21]  Georgios B. Giannakis,et al.  Distributed In-Network Channel Decoding , 2009, IEEE Transactions on Signal Processing.

[22]  Qing Ling,et al.  Decentralized Sparse Signal Recovery for Compressive Sleeping Wireless Sensor Networks , 2010, IEEE Transactions on Signal Processing.

[23]  Qing Ling,et al.  Decentralized learning for wireless communications and networking , 2015, ArXiv.

[24]  Qing Ling,et al.  On the Linear Convergence of the ADMM in Decentralized Consensus Optimization , 2013, IEEE Transactions on Signal Processing.

[25]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[26]  Seif Haridi,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[27]  L. Schuchman.  Dither Signals and Their Effect on Quantization Noise , 1964 .

[28]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[29]  Euhanna Ghadimi,et al.  Optimal Parameter Selection for the Alternating Direction Method of Multipliers (ADMM): Quadratic Problems , 2013, IEEE Transactions on Automatic Control.

[30]  John Langford,et al.  Scaling up machine learning: parallel and distributed approaches , 2011, KDD '11 Tutorials.

[31]  João M. F. Xavier,et al.  D-ADMM: A Communication-Efficient Distributed Algorithm for Separable Optimization , 2012, IEEE Transactions on Signal Processing.

[32]  D. Bertsekas,et al.  Incremental subgradient methods for nondifferentiable optimization , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[33]  Robert D. Nowak,et al.  Quantized incremental algorithms for distributed optimization , 2005, IEEE Journal on Selected Areas in Communications.

[34]  Stephen P. Boyd,et al.  A scheme for robust distributed sensor fusion based on average consensus , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[35]  Biao Chen,et al.  Distributed average consensus with bounded quantization , 2016, 2016 IEEE 17th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).