On the performance of consensus based versus Lagrangian based algorithms for quadratic cost functions

In this paper we analyze the performance of some popular algorithms used to solve distributed optimization problems involving quadratic cost functions in a multi agent system. Namely, we study the performance of standard consensus, accelerated consensus and ADMM. We analyze the scalar quadratic function case, under different scenarios and with structured graphs. We find that accelerated consensus is the algorithm with the best performance in all the cases analyzed. On the other hand, ADMM has performance comparable to the accelerated consensus when the graph is scarcely connected, while for dense graphs its performance deteriorates and becomes worse than the one of standard consensus. The results therefore suggest that the choice of the algorithm to solve the problem we analyze strongly depends on the graph, and that accelerated consensus should always be preferred.

[1]  Ιωαννησ Τσιτσικλησ,et al.  PROBLEMS IN DECENTRALIZED DECISION MAKING AND COMPUTATION , 1984 .

[2]  Henrik Sandberg,et al.  Optimal scaling of the ADMM algorithm for distributed quadratic programming , 2013, 52nd IEEE Conference on Decision and Control.

[3]  Farrokh Albuyeh,et al.  Grid of the future , 2009, IEEE Power and Energy Magazine.

[4]  Michael I. Jordan,et al.  A General Analysis of the Convergence of ADMM , 2015, ICML.

[5]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[6]  J. Hespanha,et al.  Estimation on graphs from relative measurements , 2007, IEEE Control Systems.

[7]  J. Hespanha,et al.  Estimation on Graphs from Relative Measurements Distributed algorithms and fundamental limits , 2022 .

[8]  Euhanna Ghadimi,et al.  Optimal Parameter Selection for the Alternating Direction Method of Multipliers (ADMM): Quadratic Problems , 2013, IEEE Transactions on Automatic Control.

[9]  Krzysztof C. Kiwiel,et al.  Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization , 2003, SIAM J. Optim..

[10]  Ruggero Carli,et al.  Distributed estimation via iterative projections with application to power network monitoring , 2011, Autom..

[11]  Luca Schenato,et al.  A Survey on Distributed Estimation and Control Applications Using Linear Consensus Algorithms , 2010 .

[12]  Sonia Martínez,et al.  On Distributed Convex Optimization Under Inequality and Equality Constraints , 2010, IEEE Transactions on Automatic Control.

[13]  Euhanna Ghadimi,et al.  Global convergence of the Heavy-ball method for convex optimization , 2014, 2015 European Control Conference (ECC).

[14]  P.R. Kumar,et al.  Distributed Clock Synchronization over Wireless Networks: Algorithms and Analysis , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[15]  Damiano Varagnolo,et al.  Distributed quadratic programming under asynchronous and lossy communications via Newton-Raphson consensus , 2015, 2015 European Control Conference (ECC).

[16]  Massimo Franceschetti,et al.  Random networks for communication : from statistical physics to information systems , 2008 .

[17]  Chiara Ravazzi,et al.  A distributed randomized algorithm for relative localization in sensor networks , 2013, 2013 European Control Conference (ECC).

[18]  M. Johansson,et al.  Distributed Optimization and Games: A Tutorial Overview , 2010 .

[19]  Alex Olshevsky,et al.  Linear Time Average Consensus on Fixed Graphs and Implications for Decentralized Optimization and Multi-Agent Control , 2014, 1411.4186.

[20]  John N. Tsitsiklis,et al.  Convergence Speed in Distributed Consensus and Averaging , 2009, SIAM J. Control. Optim..

[21]  Ruggero Carli,et al.  An Asynchronous Consensus-Based Algorithm for Estimation From Noisy Relative Measurements , 2014, IEEE Transactions on Control of Network Systems.

[22]  Ruggero Carli,et al.  Communication constraints in the average consensus problem , 2008, Autom..

[23]  Andreas Terzis,et al.  Accelerated corrective consensus: Converge to the exact average at a faster rate , 2011, Proceedings of the 2011 American Control Conference.

[24]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[25]  Pascal Frossard,et al.  Polynomial Filtering for Fast Convergence in Distributed Consensus , 2008, IEEE Transactions on Signal Processing.

[26]  Ruggero Carli,et al.  Multi-agents adaptive estimation and coverage control using Gaussian regression , 2015, 2015 European Control Conference (ECC).

[27]  Michael G. Rabbat,et al.  Optimization and Analysis of Distributed Averaging With Short Node Memory , 2009, IEEE Transactions on Signal Processing.

[28]  Damiano Varagnolo,et al.  Newton-Raphson Consensus for Distributed Convex Optimization , 2015, IEEE Transactions on Automatic Control.

[29]  D. Puccinelli,et al.  Wireless sensor networks: applications and challenges of ubiquitous sensing , 2005, IEEE Circuits and Systems Magazine.

[30]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[31]  S. Muthukrishnan,et al.  First- and Second-Order Diffusive Methods for Rapid, Coarse, Distributed Load Balancing , 1996, Theory of Computing Systems.

[32]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[33]  Emiliano Dall'Anese,et al.  Fast Consensus by the Alternating Direction Multipliers Method , 2011, IEEE Transactions on Signal Processing.