Factorization of completely positive matrices using iterative projected gradient steps

We aim to factorize a completely positive matrix by using an optimization approach which consists in the minimization of a nonconvex smooth function over a convex and compact set. To solve this problem we propose a projected gradient algorithm with parameters that take into account the effects of relaxation and inertia. Both projection and gradient steps are simple in the sense that they have explicit formulas and do not require inner loops. Furthermore, no expensive procedure to find an appropriate starting point is needed. The convergence analysis shows that the whole sequence of generated iterates converges to a critical point of the objective function and it makes use of the Łojasiewicz inequality. Its rate of convergence expressed in terms of the Łojasiewicz exponent of a regularization of the objective function is also provided. Numerical experiments demonstrate the efficiency of the proposed method, in particular in comparison to other factorization algorithms, and emphasize the role of the relaxation and inertial parameters.

[1]  Patrick Groetzner.  A projective approach to nonnegative matrix factorization , 2021, The Electronic Journal of Linear Algebra.

[2]  Mirjam Dür,et al.  A factorization method for completely positive matrices , 2020 .

[3]  H. Attouch,et al.  Convergence of a Relaxed Inertial Forward–Backward Algorithm for Structured Monotone Inclusions , 2019, Applied Mathematics & Optimization.

[4]  Alexandre d'Aspremont,et al.  Quartic First-Order Methods for Low-Rank Minimization , 2019, Journal of Optimization Theory and Applications.

[5]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[6]  Hedy Attouch,et al.  Convergence of a relaxed inertial proximal algorithm for maximally monotone operators , 2019, Mathematical Programming.

[7]  Matthew K. Tam,et al.  The Douglas–Rachford algorithm for convex and nonconvex feasibility problems , 2019, Mathematical Methods of Operations Research.

[8]  Chen Chen,et al.  A difference-of-convex approach for split feasibility with applications to matrix factorizations and outlier detection , 2019, Journal of Global Optimization.

[9]  Szilárd László,et al.  Convergence rates for an inertial algorithm of gradient type associated to a smooth non-convex minimization , 2018, Mathematical Programming.

[10]  Anthony Man-Cho So,et al.  Quadratic optimization with orthogonality constraint: explicit Łojasiewicz exponent and linear convergence of retraction-based line-search and stochastic variance-reduced gradient methods , 2018, Math. Program..

[11]  Radu Ioan Bot,et al.  The Proximal Alternating Direction Method of Multipliers in the Nonconvex Setting: Convergence Analysis and Rates , 2018, Math. Oper. Res..

[12]  Ting Kei Pong,et al.  Calculus of the Exponent of Kurdyka–Łojasiewicz Inequality and Its Applications to Linear Convergence of First-Order Methods , 2017, Foundations of Computational Mathematics.

[13]  Heinz H. Bauschke,et al.  Projecting onto the Intersection of a Cone and a Sphere , 2017, SIAM J. Optim..

[14]  D. Cooley,et al.  Decompositions of dependence for high-dimensional extremes , 2016, Biometrika.

[15]  Thomas Pock,et al.  Inertial Proximal Alternating Linearized Minimization (iPALM) for Nonconvex and Nonsmooth Problems , 2016, SIAM J. Imaging Sci..

[16]  Julien M. Hendrickx,et al.  A generic online acceleration scheme for optimization algorithms via relaxation and inertia , 2016, Optim. Methods Softw..

[17]  Guoyin Li,et al.  Calculus of the Exponent of Kurdyka–Łojasiewicz Inequality and Its Applications to Linear Convergence of First-Order Methods , 2016, Foundations of Computational Mathematics.

[18]  Hedy Attouch,et al.  The Rate of Convergence of Nesterov's Accelerated Forward-Backward Method is Actually Faster Than 1/k2 , 2015, SIAM J. Optim..

[19]  Immanuel M. Bomze,et al.  The structure of completely positive matrices according to their CP-rank and CP-plus-rank , 2015 .

[20]  Mirjam Dür,et al.  Open problems in the theory of completely positive and copositive matrices , 2015 .

[21]  Radu Ioan Bot,et al.  An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions , 2014, EURO J. Comput. Optim..

[22]  Guoyin Li,et al.  Douglas–Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems , 2014, Math. Program..

[23]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[24]  Radu Ioan Bot,et al.  An Inertial Tseng’s Type Proximal Algorithm for Nonsmooth and Nonconvex Optimization Problems , 2014, J. Optim. Theory Appl..

[25]  D. Szyld,et al.  Nearly positive matrices , 2014 .

[26]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[27]  Peter J. C. Dickinson,et al.  On the computational complexity of membership problems for the completely positive cone and its dual , 2014, Comput. Optim. Appl..

[28]  Mirjam Dür,et al.  Factorization and cutting planes for completely positive matrices by copositive projection , 2014, Math. Program..

[29]  Dmitriy Drusvyatskiy,et al.  Transversality and Alternating Projections for Nonconvex Sets , 2014, Found. Comput. Math..

[30]  Rina Panigrahy,et al.  Sparse Matrix Factorization , 2013, ArXiv.

[31]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[32]  Jiawang Nie,et al.  The $${\mathcal {A}}$$A-Truncated $$K$$K-Moment Problem , 2012, Found. Comput. Math..

[33]  Jiawang Nie,et al.  The A-Truncated K -Moment Problem , 2012 .

[34]  A. Berman,et al.  A characterisation of common diagonal stability over cones , 2012 .

[35]  Mirjam Dür,et al.  Linear-Time Complete Positivity Detection and Decomposition of Sparse Matrices , 2012, SIAM J. Matrix Anal. Appl..

[36]  Emmanuel J. Candès,et al.  Adaptive Restart for Accelerated Gradient Schemes , 2012, Foundations of Computational Mathematics.

[37]  Immanuel M. Bomze,et al.  Copositive optimization - Recent developments and applications , 2012, Eur. J. Oper. Res..

[38]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[39]  Florian Jarre,et al.  On the computation of C* certificates , 2009, J. Glob. Optim..

[40]  Boris Polyak,et al.  B.S. Mordukhovich. Variational Analysis and Generalized Differentiation. I. Basic Theory, II. Applications , 2009 .

[41]  Samuel Burer,et al.  On the copositive representation of binary and continuous nonconvex quadratic programs , 2009, Math. Program..

[42]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[43]  Robert Shorten,et al.  On Linear Copositive Lyapunov Functions and the Stability of Switched Positive Linear Systems , 2007, IEEE Transactions on Automatic Control.

[44]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[45]  A. Berman,et al.  Completely Positive Matrices , 2003 .

[46]  Etienne de Klerk,et al.  On Copositive Programming and Standard Quadratic Optimization Problems , 2000, J. Glob. Optim..

[47]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[48]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[49]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[50]  David Stutz.  IPIANO : INERTIAL PROXIMAL ALGORITHM FOR NON-CONVEX OPTIMIZATION , 2016 .

[51]  Changqing Xu,et al.  A simple sufficient condition for complete positivity , 2015 .

[52]  Peter J. C. Dickinson.  AN IMPROVED CHARACTERISATION OF THE INTERIOR OF THE COMPLETELY POSITIVE CONE , 2010 .

[53]  Mirjam Dür,et al.  Copositive Programming – a Survey , 2010 .

[54]  Wim Michiels,et al.  Recent Advances in Optimization and its Applications in Engineering , 2010 .

[55]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[56]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.

[57]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[58]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[59]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .