Fast decentralized gradient descent method and applications to in-situ seismic tomography

We consider the decentralized consensus optimization problem arising from in-situ seismic tomography in large-scale sensor networks. Unlike traditional seismic imaging performed in a centralized location, each node in this setting privately holds an objective function and partial data. The goal of each node is to obtain the optimal solution of the whole seismic image, while communicating only with its immediate neighbors. We present a fast decentralized gradient descent method and prove that this new method can reach optimal convergence rate of O(1/k2) where k is the number of communication/iteration rounds. Extensive numerical experiments on synthetic and real-world sensor network seismic data demonstrate that the proposed algorithms significantly outperform existing methods.

[1]  R. P. Bording,et al.  Applications of seismic travel-time tomography , 1987 .

[2]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[3]  Asuman E. Ozdaglar,et al.  On the O(1=k) convergence of asynchronous distributed alternating Direction Method of Multipliers , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[4]  Pascal Bianchi,et al.  Asynchronous distributed optimization using a randomized alternating direction method of multipliers , 2013, 52nd IEEE Conference on Decision and Control.

[5]  John S. Baras,et al.  Performance Evaluation of the Consensus-Based Distributed Subgradient Method Under Random Communication Topologies , 2011, IEEE Journal of Selected Topics in Signal Processing.

[6]  Stephen P. Boyd,et al.  A scheme for robust distributed sensor fusion based on average consensus , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[7]  Qing Ling,et al.  On the Convergence of Decentralized Gradient Descent , 2013, SIAM J. Optim..

[8]  Robert Nowak,et al.  Distributed optimization in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[9]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[10]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[11]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[12]  Qing Ling,et al.  EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization , 2014, 1404.6264.

[13]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[14]  Angelia Nedic,et al.  Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs , 2014, IEEE Transactions on Automatic Control.

[15]  Karl Henrik Johansson,et al.  Finite-time and Asymptotic Convergence of Distributed Averaging and Maximizing Algorithms , 2012, ArXiv.

[16]  John N. Tsitsiklis,et al.  Distributed asynchronous deterministic and stochastic gradient optimization algorithms , 1986 .

[17]  Per Christian Hansen,et al.  AIR Tools - A MATLAB package of algebraic iterative reconstruction methods , 2012, J. Comput. Appl. Math..

[18]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[19]  Ioannis D. Schizas,et al.  Performance Analysis of the Consensus-Based Distributed LMS Algorithm , 2009, EURASIP J. Adv. Signal Process..

[20]  Volkan Cevher,et al.  Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics , 2014, IEEE Signal Processing Magazine.

[21]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[22]  Angelia Nedic,et al.  Distributed optimization over time-varying directed graphs , 2013, 52nd IEEE Conference on Decision and Control.

[23]  R. Murray,et al.  Decentralized Multi-Agent Optimization via Dual Decomposition , 2011 .

[24]  Annie I-An Chen,et al.  Fast Distributed First-Order Methods , 2012 .

[25]  Ioannis D. Schizas,et al.  Distributed LMS for Consensus-Based In-Network Adaptive Processing , 2009, IEEE Transactions on Signal Processing.

[26]  José M. F. Moura,et al.  Fast Distributed Gradient Methods , 2011, IEEE Transactions on Automatic Control.

[27]  Alejandro Ribeiro,et al.  A distributed line search for network optimization , 2012, 2012 American Control Conference (ACC).

[28]  Anand D. Sarwate,et al.  Broadcast Gossip Algorithms for Consensus , 2009, IEEE Transactions on Signal Processing.

[29]  A.H. Sayed,et al.  Distributed Recursive Least-Squares Strategies Over Adaptive Networks , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.