Fixed point quasiconvex subgradient method

Constrained quasiconvex optimization problems appear in many fields, such as economics, engineering, and management science. In particular, fractional programming, which models ratio indicators such as the profit/cost ratio as fractional objective functions, is an important instance. Subgradient methods and their variants are useful ways for solving these problems efficiently. Many complicated constraint sets onto which it is hard to compute the metric projections in a realistic amount of time appear in these applications. This implies that the existing methods cannot be applied to quasiconvex optimization over a complicated set. Meanwhile, thanks to fixed point theory, we can construct a computable nonexpansive mapping whose fixed point set coincides with a complicated constraint set. This paper proposes an algorithm that uses a computable nonexpansive mapping for solving a constrained quasiconvex optimization problem. We provide convergence analyses for constant and diminishing step-size rules. Numerical comparisons between the proposed algorithm and an existing algorithm show that the proposed algorithm runs stably and quickly even when the running time of the existing algorithm exceeds the time limit.

[1]  H. Iiduka.  Almost sure convergence of random projected proximal and subgradient algorithms for distributed nonsmooth convex optimization , 2015, 1510.07107.

[2]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[3]  H. Yaohua,et al.  STOCHASTIC SUBGRADIENT METHOD FOR QUASI-CONVEX OPTIMIZATION PROBLEMS , 2016 .

[4]  Tong Zhang,et al.  Analysis of Multi-stage Convex Relaxation for Sparse Regularization , 2010, J. Mach. Learn. Res..

[5]  Sherwood C. Frey,et al.  Fractional Programming with Homogeneous Functions , 1974, Oper. Res..

[6]  W. R. Mann,et al.  Mean value methods in iteration , 1953 .

[7]  Z. Opial.  Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[8]  I. Yamada.  The Hybrid Steepest Descent Method for the Variational Inequality Problem over the Intersection of Fixed Point Sets of Nonexpansive Mappings , 2001 .

[9]  Martin Dyer.  Calculating surrogate constraints , 1980, Math. Program..

[10]  Hideaki Iiduka,et al.  Acceleration method for convex optimization over the fixed point set of a nonexpansive mapping , 2015, Math. Program..

[11]  M. Patriksson,et al.  Conditional subgradient optimization -- Theory and applications , 1996 .

[12]  Yiu-ming Cheung,et al.  Proximal average approximated incremental gradient descent for composite penalty regularized empirical risk minimization , 2016, Machine Learning.

[13]  C Tofallis,et al.  Fractional Programming: Theory, Methods and Applications , 1997, J. Oper. Res. Soc..

[14]  Patrick L. Combettes,et al.  Hard-constrained inconsistent signal feasibility problems , 1999, IEEE Trans. Signal Process..

[15]  Krzysztof C. Kiwiel,et al.  Convergence and efficiency of subgradient methods for quasiconvex minimization , 2001, Math. Program..

[16]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[17]  Yair Censor,et al.  Algorithms for the quasiconvex feasibility problem , 2006 .

[18]  Kazuhiro Hishinuma,et al.  Acceleration Method Combining Broadcast and Incremental Distributed Optimization Algorithms , 2014, SIAM J. Optim..

[19]  Igor V. Konnov,et al.  On Convergence Properties of a Subgradient Method , 2003, Optim. Methods Softw..

[20]  J. Penot.  Are Generalized Derivatives Sseful for Generalized Convex Functions , 1998 .

[21]  Xiaoqi Yang,et al.  Conditional subgradient methods for constrained quasi-convex optimization problems , 2016 .

[22]  Yoichi Hayashi,et al.  Optimality and convergence for convex ensemble learning with sparsity and diversity based on fixed point optimization , 2018, Neurocomputing.

[23]  Kaito Sakurai,et al.  Acceleration of the Halpern algorithm to search for a fixed point of a nonexpansive mapping , 2014 .

[24]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[25]  Timo Kuosmanen,et al.  A more efficient algorithm for Convex Nonparametric Least Squares , 2013, Eur. J. Oper. Res..

[26]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[27]  B. Halpern.  Fixed points of nonexpanding maps , 1967 .

[28]  高橋 渉.  Introduction to nonlinear and convex analysis , 2009 .

[29]  Hideaki Iiduka,et al.  Incremental subgradient method for nonsmooth convex optimization with fixed point constraints , 2016, Optim. Methods Softw..

[30]  A. Banerjee.  Convex Analysis and Optimization , 2006 .

[31]  A. A. Potapenko,et al.  Method of Successive Approximations , 1964, Encyclopedia of Evolutionary Psychological Science.

[32]  Hideaki Iiduka,et al.  Proximal point algorithms for nonsmooth convex optimization with fixed point constraints , 2016, Eur. J. Oper. Res..

[33]  Hideaki Iiduka,et al.  Convergence analysis of iterative methods for nonsmooth convex optimization over fixed point sets of quasi-nonexpansive mappings , 2015, Mathematical Programming.

[34]  Travis E. Oliphant,et al.  Guide to NumPy , 2015 .

[35]  Hideaki Iiduka,et al.  Parallel computing subgradient method for nonsmooth convex optimization over the intersection of fixed point sets of nonexpansive mappings , 2015 .

[36]  Hideaki Iiduka,et al.  Fixed Point Optimization Algorithms for Distributed Optimization in Networked Systems , 2013, SIAM J. Optim..

[37]  Xiaoqi Yang,et al.  Inexact subgradient methods for quasi-convex optimization problems , 2015, Eur. J. Oper. Res..

[38]  F. Plastria.  Lower subdifferentiable functions and their minimization by cutting planes , 1985 .