A Distributed Algorithm for Solving Linear Algebraic Equations Over Random Networks

In this paper, the problem of solving linear algebraic equations of the form $Ax=b$ among multi agents is considered. It is assumed that the interconnection graphs over which the agents communicate are random. It is assumed that each agent only knows a subset of rows of the partitioned matrix $[A,\ b]$. The problem is formulated such that this formulation does not require distribution dependency of random communication graphs. The random Krasnoselskii-Mann iterative algorithm is applied for almost sure convergence to a solution of the problem for any matrices $A$ and $b$ and any initial conditions of agents' states. The algorithm converges almost surely independently from the distribution and, therefore, is amenable to completely asynchronous operations withot B-connectivity assumption. Based on initial conditions of agents' states, we show that the limit point of the sequence generated by the algorithm is determined by the unique solution of a convex optimization problem independent from the distribution of random communication graphs.

[1]  Lili Wang,et al.  A Distributed Algorithm for Computing a Common Fixed Point of a Family of Paracontractions , 2016, ArXiv.

[2]  Ji Liu,et al.  Exponential convergence of a distributed algorithm for solving linear algebraic equations , 2017, Autom..

[3]  Kellen Petersen August Real Analysis , 2009 .

[4]  Rodrigo H. Ordóñez-Hurtado,et al.  A method for determining the non-existence of a common quadratic Lyapunov function for switched linear systems based on particle swarm optimisation , 2012, Int. J. Syst. Sci..

[5]  J. Cortés,et al.  When does a digraph admit a doubly stochastic adjacency matrix? , 2010, Proceedings of the 2010 American Control Conference.

[6]  Paul Tseng,et al.  On the Convergence of the Products of Firmly Nonexpansive Mappings , 1992, SIAM J. Optim..

[7]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Convex Optimization Over Random Networks , 2011, IEEE Transactions on Automatic Control.

[8]  Tamer Basar,et al.  A communication-efficient distributed algorithm for solving linear algebraic equations , 2014, 2014 7th International Conference on NETwork Games, COntrol and OPtimization (NetGCoop).

[9]  Jing Wang,et al.  Solving Systems of Linear Equations by Distributed Convex Optimization in the Presence of Stochastic Uncertainty , 2014 .

[10]  Minyue Fu,et al.  Distributed weighted least-squares estimation with fast convergence for large-scale systems , 2015, 52nd IEEE Conference on Decision and Control.

[11]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[12]  Dennis F. Cudia The geometry of Banach spaces , 1964 .

[13]  Rodrigo H. Ordóñez-Hurtado,et al.  A Methodology for Determining the Non-existence of Common Quadratic Lyapunov Functions for Pairs of Stable Systems , 2011, 2011 Fifth International Conference on Genetic and Evolutionary Computing.

[14]  Peng Wang,et al.  Distributed minimum weighted norm solution to linear equations associated with weighted inner product , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[15]  Tamer Basar,et al.  Stability of a distributed algorithm for solving linear algebraic equations , 2014, 53rd IEEE Conference on Decision and Control.

[16]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[17]  Brian D. O. Anderson,et al.  Distributed network flows solving linear algebraic equations , 2016, 2016 American Control Conference (ACC).

[18]  A. A. Potapenko,et al.  Method of Successive Approximations , 1964, Encyclopedia of Evolutionary Psychological Science.

[19]  Alireza Tahbaz-Salehi,et al.  Consensus Over Ergodic Stationary Graph Processes , 2010, IEEE Transactions on Automatic Control.

[20]  Peng Wang,et al.  Distributed solution to linear equations from arbitrary initializations , 2017, 2017 American Control Conference (ACC).

[21]  Shaoshuai Mou,et al.  An asynchronous distributed algorithm for solving a linear algebraic equation , 2013, 52nd IEEE Conference on Decision and Control.

[22]  Roberto Tempo,et al.  A networked parallel algorithm for solving linear algebraic equations , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[23]  Jing Wang,et al.  Distributed Least Square with intermittent communications , 2012, 2012 American Control Conference (ACC).

[24]  Lili Wang,et al.  A distributed algorithm with an arbitrary initialization for solving a linear algebraic equation , 2016, 2016 American Control Conference (ACC).

[25]  Nicola Elia,et al.  A Distributed Algorithm for Solving Linear Algebraic Equations Over Random Networks , 2021, IEEE Transactions on Automatic Control.

[26]  Asuman E. Ozdaglar,et al.  Distributed multi-agent optimization with state-dependent communication , 2010, Math. Program..

[27]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[28]  Yueqiang Shang,et al.  A distributed memory parallel Gauss-Seidel algorithm for linear algebraic systems , 2009, Comput. Math. Appl..

[29]  Chunguang Li,et al.  Distributed Sparse Recursive Least-Squares Over Networks , 2014, IEEE Transactions on Signal Processing.

[30]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[31]  A. Stephen Morse,et al.  An asynchronous distributed algorithm for computing a common fixed point of a family of paracontractions , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[32]  John M. Davis,et al.  The Positivstellensatz and nonexistence of common quadratic Lyapunov functions , 2011, 2011 IEEE 43rd Southeastern Symposium on System Theory.

[33]  Tamer Basar,et al.  Stochastic communication-efficient distributed algorithms for solving linear algebraic equations , 2016, 2016 IEEE Conference on Control Applications (CCA).

[34]  Jon Crowcroft,et al.  Parallel iterative solution method for large sparse linear equation systems , 2005 .

[35]  Choon Yik Tang,et al.  A Distributed Algorithm for Solving Positive Definite Linear Equations Over Networks With Membership Dynamics , 2018, IEEE Transactions on Control of Network Systems.

[36]  W. R. Mann,et al.  Mean value methods in iteration , 1953 .

[37]  Gonzalo Mateos,et al.  Distributed Recursive Least-Squares: Stability and Performance Analysis , 2011, IEEE Transactions on Signal Processing.

[38]  Tamer Basar,et al.  A continuous-time distributed algorithm for solving linear equations , 2016, 2016 American Control Conference (ACC).

[39]  Shaoshuai Mou,et al.  Asynchronous Distributed Algorithms for Solving Linear Algebraic Equations , 2018, IEEE Transactions on Automatic Control.

[40]  Mu Yang,et al.  A distributed algorithm for solving general linear equations over networks , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[41]  Choon Yik Tang,et al.  Distributed Asynchronous Algorithms for Solving Positive Definite Linear Equations over Networks—Part II: Wireless Networks , 2009 .

[42]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[43]  Karl Henrik Johansson,et al.  Randomized optimal consensus of multi-agent systems , 2011, Autom..

[44]  Xuan Wang,et al.  Further discussions on a distributed algorithm for solving linear algebra equations , 2017, 2017 American Control Conference (ACC).

[45]  Angelia Nedic,et al.  Asynchronous Broadcast-Based Convex Optimization Over a Network , 2011, IEEE Transactions on Automatic Control.

[46]  A. S. Morse,et al.  A distributed algorithm for efficiently solving linear equations , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[47]  Shaoshuai Mou,et al.  A distributed algorithm for efficiently solving linear equations and its applications (Special Issue JCW) , 2016, Syst. Control. Lett..

[48]  John N. Tsitsiklis,et al.  On the Nonexistence of Quadratic Lyapunov Functions for Consensus Algorithms , 2007, IEEE Transactions on Automatic Control.

[49]  Xuan Wang,et al.  Improvement of a Distributed Algorithm for Solving Linear Equations , 2017, IEEE Transactions on Industrial Electronics.

[50]  Jing Wang,et al.  Distributed solution of linear equations over unreliable networks , 2016, 2016 American Control Conference (ACC).

[51]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[52]  Shaoshuai Mou,et al.  Decentralized gradient algorithm for solution of a linear equation , 2015, ArXiv.

[53]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[54]  Shaoshuai Mou,et al.  A distributed algorithm for solving a linear algebraic equation , 2015, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[55]  Bor-Sen Chen,et al.  LaSalle-Type Theorem and Its Applications to Infinite Horizon Optimal Control of Discrete-Time Nonlinear Stochastic Systems , 2017, IEEE Transactions on Automatic Control.

[56]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.