Distributed Online Optimization for Multi-Agent Networks With Coupled Inequality Constraints

This paper investigates the distributed online optimization problem over a multi-agent network subject to local set constraints and coupled inequality constraints, which has a lot of applications in many areas, such as wireless sensor networks, power systems and plug-in electric vehicles. In this problem, the cost function at each time step is the sum of local cost functions with each of them being gradually revealed to its corresponding agent, and meanwhile only local functions in coupled inequality constraints are accessible to each agent. To address this problem, a modified primal-dual algorithm, called distributed online primal-dual push-sum algorithm (DOPP), is developed in this paper, which does not rest on any assumption on parameter boundedness and is applicable to unbalanced networks. It is shown that the proposed algorithm is sublinear for both the dynamic regret and the violation of coupled inequality constraints. Finally, the theoretical results are supported by a simulation example.

[1]  Mehran Mesbahi,et al.  Online Distributed Convex Optimization on Dynamic Networks , 2014, IEEE Transactions on Automatic Control.

[2]  Jorge Cortés,et al.  Distributed Online Convex Optimization Over Jointly Connected Digraphs , 2014, IEEE Transactions on Network Science and Engineering.

[3]  Robert Nowak,et al.  Distributed optimization in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[4]  Shahin Shahrampour,et al.  Distributed Online Optimization in Dynamic Environments Using Mirror Descent , 2016, IEEE Transactions on Automatic Control.

[5]  Maria Prandini,et al.  Dual decomposition for multi-agent distributed optimization with coupling constraints , 2016, Autom..

[6]  Hao Yu,et al.  Online Convex Optimization with Time-Varying Constraints , 2017, 1702.04783.

[7]  Manfred Morari,et al.  A decomposition method for large scale MILPs, with performance guarantees and a power system application , 2014, Autom..

[8]  Michael G. Rabbat,et al.  Push-Sum Distributed Dual Averaging for convex optimization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[9]  Alejandro Ribeiro,et al.  A Saddle Point Algorithm for Networked Online Convex Optimization , 2014, IEEE Transactions on Signal Processing.

[10]  Angelia Nedic,et al.  Distributed optimization over time-varying directed graphs , 2013, 52nd IEEE Conference on Decision and Control.

[11]  Michael M. Zavlanos,et al.  On the Sublinear Regret of Distributed Primal-Dual Algorithms for Online Constrained Optimization , 2017, 1705.11128.

[12]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[13]  John N. Tsitsiklis,et al.  Fundamental limitations for anonymous distributed systems with broadcast communications , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[14]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[15]  John N. Tsitsiklis,et al.  Weighted Gossip: Distributed Averaging using non-doubly stochastic matrices , 2010, 2010 IEEE International Symposium on Information Theory.

[16]  Cheng Wu,et al.  Distributed Convex Optimization with Inequality Constraints over Time-Varying Unbalanced Digraphs , 2016, IEEE Transactions on Automatic Control.

[17]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[18]  Karl Henrik Johansson,et al.  Distributed Online Convex Optimization With Time-Varying Coupled Inequality Constraints , 2019, IEEE Transactions on Signal Processing.

[19]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[20]  Minyue Fu,et al.  A Distributed Algorithm for Resource Allocation Over Dynamic Digraphs , 2017, IEEE Transactions on Signal Processing.

[21]  Usman A. Khan,et al.  Distributed Subgradient Projection Algorithm Over Directed Graphs , 2016, IEEE Transactions on Automatic Control.

[22]  Lihua Xie,et al.  A Bregman Splitting Scheme for Distributed Optimization Over Networks , 2018, IEEE Transactions on Automatic Control.

[23]  Lihua Xie,et al.  Distributed Continuous-Time Nonsmooth Convex Optimization With Coupled Inequality Constraints , 2020, IEEE Transactions on Control of Network Systems.

[24]  Christoforos N. Hadjicostis,et al.  Distributed strategies for average consensus in directed graphs , 2011, IEEE Conference on Decision and Control and European Control Conference.

[25]  Jorge Cortes,et al.  Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms , 2009 .

[26]  Qing Ling,et al.  An Online Convex Optimization Approach to Proactive Network Resource Allocation , 2017, IEEE Transactions on Signal Processing.

[27]  Michael M. Zavlanos,et al.  Distributed Constrained Online Learning , 2019, IEEE Transactions on Signal Processing.

[28]  Angelia Nedic,et al.  Stochastic Dual Averaging for Decentralized Online Optimization on Time-Varying Communication Graphs , 2017, IEEE Transactions on Automatic Control.

[29]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[30]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[31]  Lijun Chen,et al.  An Incentive-Based Online Optimization Framework for Distribution Grids , 2017, IEEE Transactions on Automatic Control.

[32]  Alejandro Ribeiro,et al.  Online Learning of Feasible Strategies in Unknown Environments , 2016, IEEE Transactions on Automatic Control.

[33]  Usman A. Khan,et al.  DEXTRA: A Fast Algorithm for Optimization Over Directed Graphs , 2017, IEEE Transactions on Automatic Control.

[34]  Giuseppe Notarstefano,et al.  Constraint-Coupled Distributed Optimization: A Relaxation and Duality Approach , 2017, IEEE Transactions on Control of Network Systems.

[35]  Jorge Cortés,et al.  Distributed Saddle-Point Subgradient Algorithms With Laplacian Averaging , 2015, IEEE Transactions on Automatic Control.

[36]  Bahman Gharesifard,et al.  Distributed Online Convex Optimization on Time-Varying Directed Graphs , 2017, IEEE Transactions on Control of Network Systems.

[37]  Ji Liu,et al.  Proximal Online Gradient Is Optimum for Dynamic Regret: A General Lower Bound , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[38]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[39]  Jinfeng Yi,et al.  Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient , 2016, ICML.

[40]  Daniel W. C. Ho,et al.  An Adaptive Primal-Dual Subgradient Algorithm for Online Distributed Constrained Optimization , 2018, IEEE Transactions on Cybernetics.

[41]  Sonia Martínez,et al.  On Distributed Convex Optimization Under Inequality and Equality Constraints , 2010, IEEE Transactions on Automatic Control.

[42]  Ying Xiong.  Nonlinear Optimization , 2014 .

[43]  K. J. Ray Liu,et al.  Online Convex Optimization With Time-Varying Constraints and Bandit Feedback , 2019, IEEE Transactions on Automatic Control.

[44]  Asuman E. Ozdaglar,et al.  Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods , 2008, SIAM J. Optim..

[45]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[46]  Giuseppe Notarstefano,et al.  A Duality-Based Approach for Distributed Optimization with Coupling Constraints , 2018, ArXiv.

[47]  N. S. Aybat,et al.  Distributed Linearized Alternating Direction Method of Multipliers for Composite Convex Consensus Optimization , 2015, IEEE Transactions on Automatic Control.

[48]  Angelia Nedic,et al.  Decentralized online optimization with global objectives and local communication , 2015, 2015 American Control Conference (ACC).

[49]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[50]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[51]  Angelia Nedic,et al.  Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs , 2014, IEEE Transactions on Automatic Control.

[52]  Gang Feng,et al.  Distributed Proximal Algorithms for Multiagent Optimization With Coupled Inequality Constraints , 2019, IEEE Transactions on Automatic Control.

[53]  Shahin Shahrampour,et al.  An online optimization approach for multi-agent tracking of dynamic parameters in the presence of adversarial noise , 2017, 2017 American Control Conference (ACC).

[54]  Van Sy Mai,et al.  Linear Convergence in Optimization Over Directed Graphs With Row-Stochastic Matrices , 2016, IEEE Transactions on Automatic Control.

[55]  Tamer Basar,et al.  Distributed algorithms for the computation of noncooperative equilibria , 1987, Autom..

[56]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[57]  Anna Scaglione,et al.  Distributed Constrained Optimization by Consensus-Based Primal-Dual Perturbation Method , 2013, IEEE Transactions on Automatic Control.