Asynchronous Newton-Raphson Consensus for Distributed Convex Optimization

We consider the distributed unconstrained minimization of separable convex cost functions, where the global cost is given by the sum of several local and private costs, each associated to a specific agent of a given communication network. We specifically address an asynchronous distributed optimization technique called Newton-Raphson Consensus. Beside having low computational complexity, low communication requirements and being interpretable as a distributed Newton-Raphson algorithm, the technique has also the beneficial properties of requiring very little coordination and naturally supporting time-varying topologies. In this work we analytically prove that under some assumptions it shows either local or global convergence properties, and corroborate this result by the means of numerical simulations.

[1]  M Franceschelli,et al.  Distributed Averaging in Sensor Networks Based on Broadcast Gossip Algorithms , 2011, IEEE Sensors Journal.

[2]  Stephen P. Boyd,et al.  A scheme for robust distributed sensor fusion based on average consensus , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[3]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[4]  Damiano Varagnolo,et al.  Multidimensional Newton-Raphson consensus for distributed convex optimization , 2012, 2012 American Control Conference (ACC).

[5]  Emiliano Dall'Anese,et al.  Fast Consensus by the Alternating Direction Multipliers Method , 2011, IEEE Transactions on Signal Processing.

[6]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[7]  Sandro Zampieri,et al.  Randomized consensus algorithms over large scale networks , 2007 .

[8]  Enrique Alba,et al.  A survey of parallel distributed genetic algorithms , 1999, Complex..

[9]  Ιωαννησ Τσιτσικλησ,et al.  PROBLEMS IN DECENTRALIZED DECISION MAKING AND COMPUTATION , 1984 .

[10]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[11]  Carlo Fischione,et al.  Fast-Lipschitz optimization with wireless sensor networks applications , 2011, Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks.

[12]  T. C. Aysal,et al.  Accelerated Distributed Average Consensus via Localized Node State Prediction , 2009, IEEE Transactions on Signal Processing.

[13]  Björn Johansson,et al.  On Distributed Optimization in Networked Systems , 2008 .

[14]  P. Olver.  Nonlinear Systems , 2013 .

[15]  Hassan K. Khalil,et al.  Singular perturbation methods in control : analysis and design , 1986 .

[16]  Damiano Varagnolo,et al.  Consensus‐based distributed sensor calibration and least‐square parameter identification in WSNs , 2010 .

[17]  Bart De Schutter,et al.  Particle Swarms in Optimization and Control , 2008 .

[18]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[19]  Kai Cai,et al.  Average consensus on general strongly connected digraphs , 2012, Autom..

[20]  Asuman E. Ozdaglar,et al.  A distributed newton method for network optimization , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[21]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .