Three-term conjugate gradient method for the convex optimization problem over the fixed point set of a nonexpansive mapping

Many constrained sets in problems such as signal processing and optimal control can be represented as a fixed point set of a certain nonexpansive mapping, and a number of iterative algorithms have been presented for solving a convex optimization problem over a fixed point set. This paper presents a novel gradient method with a three-term conjugate gradient direction that is used to accelerate conjugate gradient methods for solving unconstrained optimization problems. It is guaranteed that the algorithm strongly converges to the solution to the problem under the standard assumptions. Numerical comparisons with the existing gradient methods demonstrate the effectiveness and fast convergence of this algorithm.

[1]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[2]  W. A. Kirk,et al.  Topics in Metric Fixed Point Theory , 1990 .

[3]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[4]  J. Borwein,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[5]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[6]  E. Zeidler.  Nonlinear Functional Analysis and its Applications: III: Variational Methods and Optimization , 1984 .

[7]  Isao Yamada,et al.  A Use of Conjugate Gradient Direction for the Convex Optimization Problem over the Fixed Point Set of a Nonexpansive Mapping , 2008, SIAM J. Optim..

[8]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[9]  A. Goldstein.  Convex programming in Hilbert space , 1964 .

[10]  S. Hirstoaga.  Iterative selection methods for common fixed point problems , 2006 .

[11]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[12]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[13]  Ya-Xiang Yuan,et al.  Convergence properties of the Fletcher-Reeves method , 1996 .

[14]  S. Reich,et al.  Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings , 1984 .

[15]  Patrick L. Combettes,et al.  A block-iterative surrogate constraint splitting method for quadratic signal recovery , 2003, IEEE Trans. Signal Process..

[16]  Li Zhang,et al.  Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search , 2006, Numerische Mathematik.

[17]  W. Cheng.  A Two-Term PRP-Based Descent Method , 2007 .

[18]  I. Yamada.  A numerically robust hybrid steepest descent method for the convexly constrained generalized inverse problems , 2002 .

[19]  Hoon Kim,et al.  A proportional fair scheduling for multicarrier transmission systems , 2004 .

[20]  Hideaki Iiduka,et al.  Fixed point optimization algorithm and its application to power control in CDMA data networks , 2010, Mathematical Programming.

[21]  S Rich.  SOME PROBLEMS AND RESULTS IN FIXED POINT THEORY , 1983 .

[22]  Petar Popovski,et al.  Proportional fairness in multi-carrier system: upper bound and approximation algorithms , 2006, IEEE Communications Letters.

[23]  Weijun Zhou,et al.  A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence , 2006 .

[24]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[25]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[26]  Per Christian Hansen,et al.  Regularization Tools version 4.0 for Matlab 7.3 , 2007, Numerical Algorithms.

[27]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[28]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[29]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[30]  W. Takahashi.  Nonlinear Functional Analysis , 2000 .

[31]  Isao Yamada,et al.  Robust Wideband Beamforming by the Hybrid Steepest Descent Method , 2007, IEEE Transactions on Signal Processing.

[32]  Yongyi Yang,et al.  Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics , 1998 .

[33]  Li Zhang,et al.  Some descent three-term conjugate gradient methods and their global convergence , 2007, Optim. Methods Softw..

[34]  Philip Wolfe,et al.  Finding the nearest point in A polytope , 1976, Math. Program..

[35]  I. Yamada.  The Hybrid Steepest Descent Method for the Variational Inequality Problem over the Intersection of Fixed Point Sets of Nonexpansive Mappings , 2001 .