Decentralized hierarchical constrained convex optimization

This paper proposes a decentralized optimization algorithm for the triple-hierarchical constrained convex optimization problem of minimizing a sum of strongly convex functions subject to a paramonotone variational inequality constraint over an intersection of fixed point sets of nonexpansive mappings. The existing algorithms for solving this problem are centralized optimization algorithms using all the information in the problem, and these algorithms are effective, but only under certain additional restrictions. The main contribution of this paper is to present a convergence analysis of the proposed algorithm in order to show that the proposed algorithm using incremental gradients with diminishing step-size sequences converges to the solution to the problem without any additional restrictions. Another contribution of this paper is the elucidation of the practical applications of hierarchical constrained optimization in the form of network resource allocation and optimal control problems. In particular, it is shown that the proposed algorithm can be applied to decentralized network resource allocation with a triple-hierarchical structure.

[1]  F. Browder,et al.  Construction of fixed points of nonlinear mappings in Hilbert space , 1967 .

[2]  Hideaki Iiduka,et al.  Distributed Optimization for Network Resource Allocation With Nonsmooth Utility Functions , 2019, IEEE Transactions on Control of Network Systems.

[3]  W. Wonham.  On a Matrix Riccati Equation of Stochastic Control , 1968 .

[4]  Hideaki Iiduka,et al.  Computational Method for Solving a Stochastic Linear-Quadratic Control Problem Given an Unsolvable Stochastic Algebraic Riccati Equation , 2012, SIAM J. Control. Optim..

[5]  Isao Yamada,et al.  A Use of Conjugate Gradient Direction for the Convex Optimization Problem over the Fixed Point Set of a Nonexpansive Mapping , 2008, SIAM J. Optim..

[6]  Patrick L. Combettes,et al.  A block-iterative surrogate constraint splitting method for quadratic signal recovery , 2003, IEEE Trans. Signal Process..

[7]  I. Yamada.  The Hybrid Steepest Descent Method for the Variational Inequality Problem over the Intersection of Fixed Point Sets of Nonexpansive Mappings , 2001 .

[8]  Isao Yamada,et al.  Robust Wideband Beamforming by the Hybrid Steepest Descent Method , 2007, IEEE Transactions on Signal Processing.

[9]  N. Lloyd.  TOPICS IN METRIC FIXED POINT THEORY (Cambridge Studies in Advanced Mathematics 28) , 1992 .

[10]  Hideaki Iiduka,et al.  Proximal point algorithms for nonsmooth convex optimization with fixed point constraints , 2016, Eur. J. Oper. Res..

[11]  I. Yamada,et al.  Nonexpansiveness of a linearized augmented Lagrangian operator for hierarchical convex optimization , 2017 .

[12]  Hideaki Iiduka,et al.  Fixed point optimization algorithm and its application to power control in CDMA data networks , 2010, Mathematical Programming.

[13]  Hideaki Iiduka,et al.  Fixed Point Optimization Algorithms for Network Bandwidth Allocation Problems with Compoundable Constraints , 2011, IEEE Communications Letters.

[14]  Hideaki Iiduka,et al.  Convergence analysis of iterative methods for nonsmooth convex optimization over fixed point sets of quasi-nonexpansive mappings , 2015, Mathematical Programming.

[15]  Rayadurgam Srikant,et al.  The Mathematics of Internet Congestion Control , 2003 .

[16]  Roberto Cominetti,et al.  Coupling General Penalty Schemes for Convex Programming with the Steepest Descent and the Proximal Point Algorithm , 2002, SIAM J. Optim..

[17]  Xun Yu Zhou,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II , 2000, SIAM J. Control. Optim..

[18]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[19]  V. Berinde.  Iterative Approximation of Fixed Points , 2007 .

[20]  Alexandre Cabot,et al.  Proximal Point Algorithm Controlled by a Slowly Vanishing Term: Applications to Hierarchical Minimization , 2005, SIAM J. Optim..

[21]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[22]  B. Halpern.  Fixed points of nonexpanding maps , 1967 .

[23]  Alvaro R. De Pierro,et al.  Incremental Subgradients for Constrained Convex Optimization: A Unified Framework and New Methods , 2009, SIAM J. Optim..

[24]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[25]  Hideaki Iiduka,et al.  Acceleration method for convex optimization over the fixed point set of a nonexpansive mapping , 2015, Math. Program..

[26]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[27]  Asuman E. Ozdaglar,et al.  Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods , 2008, SIAM J. Optim..

[28]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[29]  W. A. Kirk,et al.  Topics in Metric Fixed Point Theory , 1990 .

[30]  Hideaki Iiduka,et al.  Fixed Point Optimization Algorithms for Distributed Optimization in Networked Systems , 2013, SIAM J. Optim..

[31]  Jean C. Walrand,et al.  Fair end-to-end window-based congestion control , 2000, TNET.

[32]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[33]  Chenshuping,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs , 1998 .

[34]  V. V. Vasin,et al.  Ill-posed problems with a priori information , 1995 .

[35]  E. Zeidler.  Nonlinear Functional Analysis and Its Applications: II/ A: Linear Monotone Operators , 1989 .

[36]  Abdellatif Moudafi,et al.  Krasnoselski–Mann iteration for hierarchical fixed-point problems , 2007 .

[37]  Paul-Emile Maingé,et al.  The viscosity approximation process for quasi-nonexpansive mappings in Hilbert spaces , 2010, Comput. Math. Appl..

[38]  W. R. Mann,et al.  Mean value methods in iteration , 1953 .

[39]  Alfredo N. Iusem,et al.  An interior point method with Bregman functions for the variational inequality problem with paramonotone operators , 1998, Math. Program..

[40]  Kazuhiro Hishinuma,et al.  Acceleration Method Combining Broadcast and Incremental Distributed Optimization Algorithms , 2014, SIAM J. Optim..

[41]  Hao Yu,et al.  A Simple Parallel Algorithm with an O(1/t) Convergence Rate for General Convex Programs , 2015, SIAM J. Optim..

[42]  K. M. Grigoriadis,et al.  Alternating convex projection methods for discrete-time covariance control design , 1996 .

[43]  Patrick L. Combettes,et al.  Hard-constrained inconsistent signal feasibility problems , 1999, IEEE Trans. Signal Process..

[44]  Hideaki Iiduka,et al.  Iterative Algorithm for Triple-Hierarchical Constrained Nonconvex Optimization Problem and Its Application to Network Bandwidth Allocation , 2012, SIAM J. Optim..

[45]  Xun Yu Zhou,et al.  Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls , 2000, IEEE Trans. Autom. Control..

[46]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[47]  Hideaki Iiduka,et al.  Iterative Algorithm for Solving Triple-Hierarchical Constrained Optimization Problem , 2011, J. Optim. Theory Appl..

[48]  J. Baillon,et al.  Quelques propriétés des opérateurs angle-bornés etn-cycliquement monotones , 1977 .

[49]  R. Wittmann.  Approximation of fixed points of nonexpansive mappings , 1992 .

[50]  Frank Kelly,et al.  Charging and rate control for elastic traffic , 1997, Eur. Trans. Telecommun..

[51]  Paul-Emile Maingé,et al.  Strong convergence of an iterative method for hierarchical fixed point problems , 2007 .

[52]  Hédy Attouch,et al.  Viscosity Solutions of Minimization Problems , 1996, SIAM J. Optim..

[53]  W. Takahashi.  Nonlinear Functional Analysis , 2000 .