Acceleration Method Combining Broadcast and Incremental Distributed Optimization Algorithms

This paper considers a networked system consisting of an operator, which manages the system, and a finite number of subnetworks with all users, and studies the problem of minimizing the sum of the operator's and all users' objective functions over the intersection of the operator's and all users' constraint sets. When users in each subnetwork can communicate with each other, they can implement an incremental subgradient method that uses the transmitted information from their neighbor users. Since the operator can communicate with users in the subnetworks, it can implement a broadcast distributed algorithm that uses all available information in the subnetworks. We present an iterative method combining broadcast and incremental distributed optimization algorithms. Our method has faster convergence and a wider range of application compared with conventional distributed algorithms. We also prove that under certain assumptions our method converges to the solution to the problem in the sense of the strong topol...

[1]  Patrick L. Combettes,et al.  Hard-constrained inconsistent signal feasibility problems , 1999, IEEE Trans. Signal Process..

[2]  Patrick Maillé,et al.  Managing a Peer-to-Peer Data Storage System in a Selfish Society , 2008, IEEE Journal on Selected Areas in Communications.

[3]  Isao Yamada,et al.  A Use of Conjugate Gradient Direction for the Convex Optimization Problem over the Fixed Point Set of a Nonexpansive Mapping , 2008, SIAM J. Optim..

[4]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[5]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[6]  Hideaki Iiduka,et al.  Acceleration method for convex optimization over the fixed point set of a nonexpansive mapping , 2015, Math. Program..

[7]  Patrick L. Combettes,et al.  A block-iterative surrogate constraint splitting method for quadratic signal recovery , 2003, IEEE Trans. Signal Process..

[8]  Hideaki Iiduka,et al.  Fixed point optimization algorithm and its application to network bandwidth allocation , 2012, J. Comput. Appl. Math..

[9]  D. Bertsekas,et al.  Incremental subgradient methods for nondifferentiable optimization , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[10]  Krzysztof C. Kiwiel,et al.  Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization , 2003, SIAM J. Optim..

[11]  I. Yamada.  The Hybrid Steepest Descent Method for the Variational Inequality Problem over the Intersection of Fixed Point Sets of Nonexpansive Mappings , 2001 .

[12]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[13]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[14]  Petar Popovski,et al.  Proportional fairness in multi-carrier system: upper bound and approximation algorithms , 2006, IEEE Communications Letters.

[15]  Demosthenis Teneketzis,et al.  An Externalities-Based Decentralized Optimal Power Allocation Algorithm for Wireless Networks , 2009, IEEE/ACM Transactions on Networking.

[16]  Alfred O. Hero,et al.  A Convergent Incremental Gradient Method with a Constant Step Size , 2007, SIAM J. Optim..

[17]  V. Berinde.  Iterative Approximation of Fixed Points , 2007 .

[18]  P. L. Combettes.  Iterative construction of the resolvent of a sum of maximal monotone operators , 2009 .

[19]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[20]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[21]  Hideaki Iiduka,et al.  Fixed Point Optimization Algorithms for Distributed Optimization in Networked Systems , 2013, SIAM J. Optim..

[22]  Hideaki Iiduka,et al.  Fixed point optimization algorithm and its application to power control in CDMA data networks , 2010, Mathematical Programming.

[23]  Mikael Johansson,et al.  A Randomized Incremental Subgradient Method for Distributed Optimization in Networked Systems , 2009, SIAM J. Optim..

[24]  Z. Opial.  Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .