SUBGRADIENT PROJECTION OVER DIRECTED GRAPHS USING SURPLUS CONSENSUS

In this paper, we propose Directed-Distributed Projected Sub-gradient (D-DPS) to solve a distributed constrained optimization problem over a sensor network. Sensors collaboratively minimize a sum of convex functions, which are only locally known and constrained to some commonly known convex set. D-DPS is based on surplus consensus, which overcomes the information asymmetry caused by directed communication and has a convergence rate of $O\left ({ {\frac {\ln k}{\sqrt {k} }} }\right )$.

[1]  Francisco Facchinei,et al.  Parallel and Distributed Methods for Constrained Nonconvex Optimization—Part I: Theory , 2016, IEEE Transactions on Signal Processing.

[2]  Yurii Nesterov,et al.  Random Block Coordinate Descent Methods for Linearly Constrained Optimization over Networks , 2015, J. Optim. Theory Appl..

[3]  Usman A. Khan,et al.  Distributed Subgradient Projection Algorithm Over Directed Graphs , 2016, IEEE Transactions on Automatic Control.

[4]  Waheed Uz Zaman Bajwa,et al.  Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data , 2014, IEEE Transactions on Signal Processing.

[5]  Na Li,et al.  Harnessing smoothness to accelerate distributed optimization , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[6]  Usman A. Khan,et al.  An Opportunistic Linear–Convex Algorithm for Localization in Mobile Robot Networks , 2017, IEEE Transactions on Robotics.

[7]  Soummya Kar,et al.  DILAND: An Algorithm for Distributed Sensor Localization With Noisy Distance Measurements , 2009, IEEE Transactions on Signal Processing.

[8]  Gesualdo Scutari,et al.  NEXT: In-Network Nonconvex Optimization , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[9]  Volkan Cevher,et al.  Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics , 2014, IEEE Signal Processing Magazine.

[10]  João M. F. Xavier,et al.  D-ADMM: A Communication-Efficient Distributed Algorithm for Separable Optimization , 2012, IEEE Transactions on Signal Processing.

[11]  Usman A. Khan,et al.  ADD-OPT: Accelerated Distributed Directed Optimization , 2016, IEEE Transactions on Automatic Control.

[12]  Ion Necoara,et al.  Random Coordinate Descent Algorithms for Multi-Agent Convex Optimization Over Networks , 2013, IEEE Transactions on Automatic Control.

[13]  Usman A. Khan,et al.  DEXTRA: A Fast Algorithm for Optimization Over Directed Graphs , 2017, IEEE Transactions on Automatic Control.

[14]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[15]  Asuman E. Koksal,et al.  Graph balancing for distributed subgradient methods over directed graphs , 2016 .

[16]  Angelia Nedic,et al.  Distributed Optimization Over Time-Varying Directed Graphs , 2015, IEEE Trans. Autom. Control..

[17]  Kai Cai,et al.  Average consensus on general strongly connected digraphs , 2012, Autom..

[18]  Usman A. Khan,et al.  On the distributed optimization over directed networks , 2015, Neurocomputing.

[19]  Qing Ling,et al.  EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization , 2014, 1404.6264.

[20]  Van Sy Mai,et al.  Linear convergence in directed optimization with row-stochastic matrices , 2016 .

[21]  Usman A. Khan,et al.  Distributed Estimation Recovery Under Sensor Failure , 2017, IEEE Signal Processing Letters.

[22]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[23]  Pascal Bianchi,et al.  A stochastic coordinate descent primal-dual algorithm and applications , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[24]  José M. F. Moura,et al.  Linear Convergence Rate of a Class of Distributed Augmented Lagrangian Algorithms , 2013, IEEE Transactions on Automatic Control.

[25]  Usman A. Khan,et al.  Graph-Theoretic Distributed Inference in Social Networks , 2014, IEEE Journal of Selected Topics in Signal Processing.

[26]  Qing Ling,et al.  On the Convergence of Decentralized Gradient Descent , 2013, SIAM J. Optim..