A Distributed Algorithm for Least Squares Solutions

In this technical note, a distributed algorithm is proposed for multiagent networks to achieve a least squares solution of a system of linear equations, in which each agent only knows part of the overall equations and communicates only with its nearby neighbors. The proposed algorithm is discrete time but does not involve small or time-varying step sizes. Given that the network is fixed, connected, and undirected, the proposed algorithm enables all agents in the network to achieve exponentially fast the same least squares solution; this is validated by simulations.

[1]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[2]  James T. Kwok,et al.  Asynchronous Distributed ADMM for Consensus Optimization , 2014, ICML.

[3]  Brian D. O. Anderson,et al.  Network Flows That Solve Linear Equations , 2015, IEEE Transactions on Automatic Control.

[4]  Shaoshuai Mou,et al.  A distributed algorithm for efficiently solving linear equations and its applications (Special Issue JCW) , 2016, Syst. Control. Lett..

[5]  Ji Liu,et al.  Exponential convergence of a distributed algorithm for solving linear algebraic equations , 2017, Autom..

[6]  Tilo Strutz,et al.  Data Fitting and Uncertainty: A practical introduction to weighted least squares and beyond , 2010 .

[7]  Lili Wang,et al.  A distributed algorithm with an arbitrary initialization for solving a linear algebraic equation , 2016, 2016 American Control Conference (ACC).

[8]  Jing Wang,et al.  A control perspective for centralized and distributed convex optimization , 2011, IEEE Conference on Decision and Control and European Control Conference.

[9]  Choon Yik Tang,et al.  A Distributed Algorithm for Solving Positive Definite Linear Equations Over Networks With Membership Dynamics , 2018, IEEE Transactions on Control of Network Systems.

[10]  Jing Wang,et al.  Control approach to distributed optimization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[11]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[12]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[13]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[14]  Xuan Wang,et al.  Improvement of a Distributed Algorithm for Solving Linear Equations , 2017, IEEE Transactions on Industrial Electronics.

[15]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[16]  Jianhua Zhang,et al.  Distributed Optimization Algorithms for Wide-Area Oscillation Monitoring in Power Systems Using Interregional PMU-PDC Architectures , 2015, IEEE Transactions on Smart Grid.

[17]  Shaoshuai Mou,et al.  A Distributed Algorithm for Solving a Linear Algebraic Equation , 2013, IEEE Transactions on Automatic Control.

[18]  José M. F. Moura,et al.  Fast Distributed Gradient Methods , 2011, IEEE Transactions on Automatic Control.

[19]  Asuman E. Ozdaglar,et al.  Distributed Alternating Direction Method of Multipliers , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[20]  Anna Scaglione,et al.  Distributed Constrained Optimization by Consensus-Based Primal-Dual Perturbation Method , 2013, IEEE Transactions on Automatic Control.

[21]  Jing Wang,et al.  Distributed Least Square with intermittent communications , 2012, 2012 American Control Conference (ACC).

[22]  Francis A. Longstaff,et al.  Valuing American Options by Simulation: A Simple Least-Squares Approach , 2001 .

[23]  Anit Kumar Sahu,et al.  Distributed Constrained Recursive Nonlinear Least-Squares Estimation: Algorithms and Asymptotics , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[24]  Jing Wang,et al.  Distributed solution of linear equations over unreliable networks , 2016, 2016 American Control Conference (ACC).

[25]  Shaoshuai Mou,et al.  A fixed-neighbor, distributed algorithm for solving a linear algebraic equation , 2013, 2013 European Control Conference (ECC).

[26]  Ioannis D. Schizas,et al.  Distributed Recursive Least-Squares for Consensus-Based In-Network Adaptive Estimation , 2009, IEEE Transactions on Signal Processing.

[27]  Shaoshuai Mou,et al.  Decentralized gradient algorithm for solution of a linear equation , 2015, ArXiv.

[28]  Steven J. Ruuth,et al.  Implicit-explicit Runge-Kutta methods for time-dependent partial differential equations , 1997 .

[29]  Bahman Gharesifard,et al.  Distributed Continuous-Time Convex Optimization on Weight-Balanced Digraphs , 2012, IEEE Transactions on Automatic Control.

[30]  E. Jury.  A Simplified Stability Criterion for Linear Discrete Systems , 1962, Proceedings of the IRE.