A randomized primal distributed algorithm for partitioned and big-data non-convex optimization

In this paper we consider a distributed optimization scenario in which the aggregate objective function to minimize is partitioned, big-data and possibly non-convex. Specifically, we focus on a set-up in which the dimension of the decision variable depends on the network size as well as the number of local functions, but each local function handled by a node depends only on a (small) portion of the entire optimization variable. This problem set-up has been shown to appear in many interesting network application scenarios. As main paper contribution, we develop a simple, primal distributed algorithm to solve the optimization problem, based on a randomized descent approach, which works under asynchronous gossip communication. We prove that the proposed asynchronous algorithm is a proper, ad-hoc version of a coordinate descent method and thus converges to a stationary point. To show the effectiveness of the proposed algorithm, we also present numerical simulations on a non-convex quadratic program, which confirm the theoretical results.

[1]  Peter Richtárik,et al.  Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function , 2011, Mathematical Programming.

[2]  João M. F. Xavier,et al.  Distributed Optimization With Local Domains: Applications in MPC and Network Flows , 2013, IEEE Transactions on Automatic Control.

[3]  Asuman E. Ozdaglar,et al.  On the O(1=k) convergence of asynchronous distributed alternating Direction Method of Multipliers , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[4]  Giuseppe Notarstefano,et al.  Randomized dual proximal gradient for large-scale distributed optimization , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[5]  Damiano Varagnolo,et al.  Asynchronous Newton-Raphson Consensus for Distributed Convex Optimization , 2012 .

[6]  Frank L. Lewis,et al.  A Distributed Auction-Based Algorithm for the Nonconvex Economic Dispatch Problem , 2014, IEEE Transactions on Industrial Informatics.

[7]  Ion Necoara,et al.  Random Coordinate Descent Algorithms for Multi-Agent Convex Optimization Over Networks , 2013, IEEE Transactions on Automatic Control.

[8]  Ruggero Carli,et al.  A Robust Block-Jacobi Algorithm for Quadratic Programming under Lossy Communications , 2015 .

[9]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[10]  Francisco Facchinei,et al.  Parallel Selective Algorithms for Nonconvex Big Data Optimization , 2014, IEEE Transactions on Signal Processing.

[11]  Pascal Bianchi,et al.  A stochastic primal-dual algorithm for distributed asynchronous composite optimization , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[12]  Karl Henrik Johansson,et al.  Distributed Event-Triggered Control for Multi-Agent Systems , 2012, IEEE Transactions on Automatic Control.

[13]  Ion Necoara,et al.  Efficient random coordinate descent algorithms for large-scale structured nonconvex optimization , 2013, Journal of Global Optimization.

[14]  Gesualdo Scutari,et al.  NEXT: In-Network Nonconvex Optimization , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[15]  Ruggero Carli,et al.  Distributed partition-based optimization via dual decomposition , 2013, 52nd IEEE Conference on Decision and Control.

[16]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[17]  Angelia Nedic,et al.  Asynchronous Gossip-Based Random Projection Algorithms Over Networks , 2013, IEEE Transactions on Automatic Control.

[18]  Peter Richtárik,et al.  Parallel coordinate descent methods for big data optimization , 2012, Mathematical Programming.

[19]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[20]  Tomaso Erseghe,et al.  A Distributed and Scalable Processing Method Based Upon ADMM , 2012, IEEE Signal Processing Letters.

[21]  Ion Necoara,et al.  Parallel Random Coordinate Descent Method for Composite Minimization: Convergence Analysis and Error Bounds , 2016, SIAM J. Optim..