Incremental quasi-subgradient methods for minimizing the sum of quasi-convex functions

The sum of ratios problem has a variety of important applications in economics and management science, but it is difficult to globally solve this problem. In this paper, we consider the minimization problem of the sum of a number of nondifferentiable quasi-convex component functions over a closed and convex set. The sum of quasi-convex component functions is not necessarily to be quasi-convex, and so, this study goes beyond quasi-convex optimization. Exploiting the structure of the sum-minimization problem, we propose a new incremental quasi-subgradient method for this problem and investigate its convergence properties to a global optimal value/solution when using the constant, diminishing or dynamic stepsize rules and under a homogeneous assumption and the Holder condition. To economize on the computation cost of subgradients of a large number of component functions, we further propose a randomized incremental quasi-subgradient method, in which only one component function is randomly selected to construct the subgradient direction at each iteration. The convergence properties are obtained in terms of function values and iterates with probability 1. The proposed incremental quasi-subgradient methods are applied to solve the quasi-convex feasibility problem and the sum of ratios problem, as well as the multiple Cobb–Douglas productions efficiency problem, and the numerical results show that the proposed methods are efficient for solving the large-scale sum of ratios problem.

[1]  H. Yaohua,et al.  STOCHASTIC SUBGRADIENT METHOD FOR QUASI-CONVEX OPTIMIZATION PROBLEMS , 2016 .

[2]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[3]  Krzysztof C. Kiwiel,et al.  Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization , 2003, SIAM J. Optim..

[4]  Sherwood C. Frey,et al.  Fractional Programming with Homogeneous Functions , 1974, Oper. Res..

[5]  Angelia Nedic,et al.  Incremental Stochastic Subgradient Algorithms for Convex Optimization , 2008, SIAM J. Optim..

[6]  Xiaoqi Yang,et al.  A Unified Augmented Lagrangian Approach to Duality and Exact Penalization , 2003, Math. Oper. Res..

[7]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[8]  Siegfried Schaible,et al.  Handbook of Generalized Convexity and Generalized Monotonicity , 2005 .

[9]  Roland W. Freund,et al.  Solving the Sum-of-Ratios Problem by an Interior-Point Method , 2001, J. Glob. Optim..

[10]  Lin Xiao,et al.  A Proximal Stochastic Gradient Method with Progressive Variance Reduction , 2014, SIAM J. Optim..

[11]  Mikael Johansson,et al.  A Randomized Incremental Subgradient Method for Distributed Optimization in Networked Systems , 2009, SIAM J. Optim..

[12]  Didier Aussel,et al.  Normal Characterization of the Main Classes of Quasiconvex Functions , 2000 .

[13]  Alan F. Blackwell,et al.  Programming , 1973, CSC '73.

[14]  Siegfried Schaible,et al.  Fractional programming: The sum-of-ratios case , 2003, Optim. Methods Softw..

[15]  Alvaro R. De Pierro,et al.  Incremental Subgradients for Constrained Convex Optimization: A Unified Framework and New Methods , 2009, SIAM J. Optim..

[16]  Robert D. Nowak,et al.  Quantized incremental algorithms for distributed optimization , 2005, IEEE Journal on Selected Areas in Communications.

[17]  Harold P. Benson Branch-and-Bound Outer Approximation Algorithm for Sum-of-Ratios Fractional Programs , 2010 .

[18]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[19]  Y. Ye,et al.  On the Complexity of a Column Generation Algorithm for Convex or Quasiconvex Feasibility Problems , 1994 .

[20]  Narin Petrot,et al.  Adaptive subgradient method for the split quasi-convex feasibility problems , 2016 .

[21]  Dimitri P. Bertsekas,et al.  Convex Optimization Algorithms , 2015 .

[22]  Hiroshi Konno,et al.  BOND PORTFOLIO OPTIMIZATION BY BILINEAR FRACTIONAL PROGRAMMING , 1989 .

[23]  Xiaoqi Yang,et al.  Conditional subgradient methods for constrained quasi-convex optimization problems , 2016 .

[24]  Igor V. Konnov,et al.  On Convergence Properties of a Subgradient Method , 2003, Optim. Methods Softw..

[25]  S. Schaible,et al.  4. Application of Generalized Concavity to Economics , 1988 .

[26]  Krzysztof C. Kiwiel,et al.  Convergence and efficiency of subgradient methods for quasiconvex minimization , 2001, Math. Program..

[27]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[28]  M. Patriksson,et al.  Conditional subgradient optimization -- Theory and applications , 1996 .

[29]  Dimitri P. Bertsekas,et al.  The effect of deterministic noise in subgradient methods , 2010, Math. Program..

[30]  Xiaoqi Yang,et al.  Abstract convergence theorem for quasi-convex optimization problems with applications , 2019 .

[31]  I. M. Stancu-Minasian Integer and Mixed Integer Linear Fractional Programming , 1997 .

[32]  D. Bertsekas,et al.  Incremental subgradient methods for nondifferentiable optimization , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[33]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[34]  Marc Teboulle,et al.  Projected subgradient methods with non-Euclidean distances for non-differentiable convex minimization and variational inequalities , 2009, Math. Program..

[35]  Yu. M. Ermol’ev Methods of solution of nonlinear extremal problems , 1966 .

[36]  Julien Mairal,et al.  Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning , 2014, SIAM J. Optim..

[37]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[38]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[39]  Xiaoqi Yang,et al.  Inexact subgradient methods for quasi-convex optimization problems , 2015, Eur. J. Oper. Res..

[40]  Chong Li,et al.  On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications , 2016, SIAM J. Optim..

[41]  I. V. Konnov On properties of supporting and quasi-supporting vectors , 1994 .

[42]  Shiqian Ma,et al.  Barzilai-Borwein Step Size for Stochastic Gradient Descent , 2016, NIPS.

[43]  Chen He,et al.  Normalized Incremental Subgradient Algorithm and Its Application , 2009, IEEE Transactions on Signal Processing.

[44]  Yair Censor,et al.  Algorithms for the quasiconvex feasibility problem , 2006 .

[45]  J. Martínez-Legaz,et al.  Generalized Convexity, Generalized Monotonicity: Recent Results , 2011 .