Adaptive First-Order Methods for General Sparse Inverse Covariance Selection

In this paper we consider estimating sparse inverse covariance of a Gaussian graphical model whose conditional independence is assumed to be partially known. Similarly as in [A. d'Aspremont, O. Banerjee, and L. El Ghaoui, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 56-66; M. Yuan and Y. Lin, Biometrika, 94 (2007), pp. 19-35], we formulate it as an $l_1$-norm penalized maximum likelihood estimation problem. Further, we propose an algorithm framework, and develop two first-order methods, that is, the adaptive spectral projected gradient (ASPG) method and the adaptive Nesterov's smooth (ANS) method, for solving this estimation problem. Finally, we compare the performance of these two methods with glasso [J. Friedman, T. Hastie, and R. Tibshirani, Biostatistics, 9 (2008), pp. 432-441; J. Friedman, T. Hastie, and R. Tibshirani, Glasso: Graphical Lasso for R, Software package, Department of Statistics, Stanford University, Stanford, CA, 2007] on a set of randomly generated instances. Our computational results demonstrate that our methods are capable of solving problems of size at least a thousand and number of constraints of nearly a half million within a reasonable amount of time, and moreover, that the ASPG method generally outperforms the ANS method and glasso.

[1]  Vwani P. Roychowdhury,et al.  Covariance selection for nonchordal graphs via chordal embedding , 2008, Optim. Methods Softw..

[2]  M. West,et al.  Sparse graphical models for exploring gene expression data , 2004 .

[3]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[4]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[5]  Stephen P. Boyd,et al.  Determinant Maximization with Linear Matrix Inequality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[6]  Alexandre d'Aspremont,et al.  First-Order Methods for Sparse Covariance Selection , 2006, SIAM J. Matrix Anal. Appl..

[7]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[8]  Jeff A. Bilmes,et al.  Factored sparse inverse covariance matrices , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[9]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[10]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[11]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[12]  Zhaosong Lu,et al.  Smooth Optimization Approach for Sparse Covariance Selection , 2008, SIAM J. Optim..

[13]  M. Raydan,et al.  Spectral Projected Gradient Methods , 2007 .

[14]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[15]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[16]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[17]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[18]  O. SIAMJ..  SMOOTH OPTIMIZATION APPROACH FOR SPARSE COVARIANCE SELECTION∗ , 2009 .