A class of modified FR conjugate gradient method and applications to non-negative matrix factorization

Conjugate gradient method, as an efficient method, is used to solve unconstrained optimization problems. In this paper, we propose a class of modified FletcherReeves conjugate gradient method, with Armijo-type line search, which generates the direction is descent for the objective function. Under mild conditions, we give that the proposed method is descent and globally convergent. Moreover, the method is applied to nonnegative matrix factorization, and the experimental results demonstrate the validity of the proposed method.

[1]  L. Chambers.  Practical methods of optimization (2nd edn) , by R. Fletcher. Pp. 436. £34.95. 2000. ISBN 0 471 49463 1 (Wiley). , 2001, The Mathematical Gazette.

[2]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[3]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[4]  Yunde Jia,et al.  Non-negative matrix factorization framework for face recognition , 2005, Int. J. Pattern Recognit. Artif. Intell..

[5]  Hongwei Liu,et al.  Modified subspace Barzilai-Borwein gradient method for non-negative matrix factorization , 2013, Comput. Optim. Appl..

[6]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[7]  Liu Guanghui,et al.  Global convergence of the fletcher-reeves algorithm with inexact linesearch , 1995 .

[8]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[9]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[10]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[11]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[12]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[13]  Li Zhang,et al.  Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search , 2006, Numerische Mathematik.