A Smooth Double Proximal Primal-Dual Algorithm for a Class of Distributed Nonsmooth Optimization Problems

This technical note studies a class of distributed nonsmooth convex consensus optimization problems. The cost function is a summation of local cost functions which are convex but nonsmooth. Each of the local cost functions consists of a twice differentiable (smooth) convex function and two lower semi-continuous (nonsmooth) convex functions. We call these problems as single-smooth plus double-nonsmooth (SSDN) problems. Under mild conditions, we propose a distributed double proximal primal-dual optimization algorithm. Double proximal splitting is designed to deal with the difficulty caused by the unproximable property of the summation of those two nonsmooth functions. Besides, it can also guarantee that the proposed algorithm is locally Lipschitz continuous. An auxiliary variable in the double proximal splitting is introduced to estimate the subgradient of the second nonsmooth function. Theoretically, we conduct the convergence analysis by employing Lyapunov stability theory. It shows that the proposed algorithm can make the states achieve consensus at the optimal point. In the end, nontrivial simulations are presented and the results demonstrate the effectiveness of the proposed algorithm.

[1]  Yiguang Hong,et al.  Distributed Continuous-Time Algorithm for Constrained Convex Optimizations via Nonsmooth Analysis Approach , 2015, IEEE Transactions on Automatic Control.

[2]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[3]  Jay A. Farrell,et al.  Distributed Continuous-Time Optimization: Nonuniform Gradient Gains, Finite-Time Convergence, and Convex Constraint Set , 2017, IEEE Transactions on Automatic Control.

[4]  N. S. Aybat,et al.  Distributed Linearized Alternating Direction Method of Multipliers for Composite Convex Consensus Optimization , 2015, IEEE Transactions on Automatic Control.

[5]  Qing Ling,et al.  A Proximal Gradient Algorithm for Decentralized Composite Optimization , 2015, IEEE Transactions on Signal Processing.

[6]  Michael M. Zavlanos,et al.  Approximate Projection Methods for Decentralized Optimization With Functional Constraints , 2015, IEEE Transactions on Automatic Control.

[7]  Anna Scaglione,et al.  Distributed Constrained Optimization by Consensus-Based Primal-Dual Perturbation Method , 2013, IEEE Transactions on Automatic Control.

[8]  Jing Wang,et al.  Control approach to distributed optimization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[9]  Lihua Xie,et al.  Distributed constrained optimal consensus of multi-agent systems , 2016, Autom..

[10]  Ernest K. Ryu,et al.  Proximal-Proximal-Gradient Method , 2017, Journal of Computational Mathematics.

[11]  Ming Yan,et al.  A new primal-dual algorithm for minimizing the sum of three functions with a linear operator , 2016, 1611.09805.

[12]  Gauthier Gidel,et al.  Adaptive Three Operator Splitting , 2018, ICML.

[13]  A. Ruszczynski,et al.  Nonlinear Optimization , 2006 .

[14]  Bahman Gharesifard,et al.  Distributed Continuous-Time Convex Optimization on Weight-Balanced Digraphs , 2012, IEEE Transactions on Automatic Control.

[15]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[16]  Mihailo R. Jovanovic,et al.  The Proximal Augmented Lagrangian Method for Nonsmooth Composite Optimization , 2016, IEEE Transactions on Automatic Control.

[17]  Shu Liang,et al.  Distributed Nonsmooth Optimization With Coupled Inequality Constraints via Modified Lagrangian Function , 2016, IEEE Transactions on Automatic Control.

[18]  Wei Shi,et al.  A Decentralized Proximal-Gradient Method With Network Independent Step-Sizes and Separated Convergence Rates , 2017, IEEE Transactions on Signal Processing.

[19]  Qingshan Liu,et al.  A Second-Order Multi-Agent Network for Bound-Constrained Distributed Optimization , 2015, IEEE Transactions on Automatic Control.

[20]  Jie Chen,et al.  Flocking of Second-Order Multiagent Systems With Connectivity Preservation Based on Algebraic Connectivity Estimation , 2017, IEEE Transactions on Cybernetics.

[21]  Nikolaos S. Papageorgiou,et al.  An introduction to nonlinear analysis , 2003 .

[22]  Feng Liu,et al.  Initialization-free distributed algorithms for optimal resource allocation with feasibility constraints and application to economic dispatch of power systems , 2015, Autom..

[23]  Jorge Cortés,et al.  Distributed Coordination for Nonsmooth Convex Optimization via Saddle-Point Dynamics , 2016, J. Nonlinear Sci..

[24]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[25]  Jorge Cortés,et al.  Distributed Saddle-Point Subgradient Algorithms With Laplacian Averaging , 2015, IEEE Transactions on Automatic Control.

[26]  Mingyi Hong,et al.  Stochastic Proximal Gradient Consensus Over Random Networks , 2015, IEEE Transactions on Signal Processing.

[27]  Damek Davis,et al.  A Three-Operator Splitting Scheme and its Optimization Applications , 2015, 1504.01032.

[28]  Zdzisław Denkowski,et al.  An Introduction to Nonlinear Analysis: Theory , 2013 .

[29]  Qingshan Liu,et al.  A Multi-Agent System With a Proportional-Integral Protocol for Distributed Constrained Optimization , 2017, IEEE Transactions on Automatic Control.

[30]  Shengyuan Xu,et al.  Distributed Primal–Dual Subgradient Method for Multiagent Optimization via Consensus Algorithms , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Feng Liu,et al.  Distributed gradient algorithm for constrained optimization with application to load sharing in power systems , 2015, Syst. Control. Lett..