A multi-block alternating direction method with parallel splitting for decentralized consensus optimization

Decentralized optimization has attracted much research interest for resource-limited networked multi-agent systems in recent years. Decentralized Tconsensus optimization, which is one of the decentralized optimization problems of great practical importance, minimizes an objective function that is the sum of the terms from individual agents over a set of variables on which all the agents should reach a consensus. This problem can be reformulated into an equivalent model with two blocks of variables, which can then be solved by the alternating direction method (ADM) with only communications between neighbor nodes. Motivated by a recently emerged class of so-called multi-block ADMs, this article demonstrates that it is more natural to reformulate a decentralized consensus optimization problem to one with multiple blocks of variables and solve it by a multi-block ADM. In particular, we focus on the multi-block ADM with parallel splitting, which has easy decentralized implementation. Convergence rate is analyzed in the setting of average consensus, and the relation between two-block and multi-block ADMs are studied. Numerical experiments demonstrate the effectiveness of the multi-block ADM with parallel splitting in terms of speed and communication cost and show that it has better network scalability.