Efficient Implementations of the Generalized Lasso Dual Path Algorithm

We consider efficient implementations of the generalized lasso dual path algorithm given by Tibshirani and Taylor in 2011. We first describe a generic approach that covers any penalty matrix D and any (full column rank) matrix X of predictor variables. We then describe fast implementations for the special cases of trend filtering problems, fused lasso problems, and sparse fused lasso problems, both with X = I and a general matrix X. These specialized implementations offer a considerable improvement over the generic implementation, both in terms of numerical stability and efficiency of the solution path computation. These algorithms are all available for use in the genlasso R package, which can be found in the CRAN repository.

[1]  Antonin Chambolle,et al.  On Total Variation Minimization and Surface Evolution Using Parametric Maximum Flows , 2009, International Journal of Computer Vision.

[2]  Timothy A. Davis,et al.  Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization , 2011, TOMS.

[3]  S. Rosset,et al.  Piecewise linear regularized solution paths , 2007, 0708.2197.

[4]  Holger Hoefling.  A Path Algorithm for the Fused Lasso Signal Approximator , 2009, 0910.0526.

[5]  M. R. Osborne,et al.  On the LASSO and its Dual , 2000 .

[6]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[7]  K. Lange,et al.  A Path Algorithm for Constrained Estimation , 2011, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[8]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[9]  Nicholas A. Johnson,et al.  A Dynamic Programming Algorithm for the Fused Lasso and L 0-Segmentation , 2013 .

[10]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[11]  Martin Berggren,et al.  Hybrid differentiation strategies for simulation and analysis of applications in C++ , 2008, TOMS.

[12]  Stephan Didas,et al.  Splines in Higher Order TV Regularization , 2006, International Journal of Computer Vision.

[13]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[14]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[15]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[16]  R. Tibshirani,et al.  The solution path of the generalized lasso , 2010, 1005.1971.

[17]  P. Davies,et al.  Local Extremes, Runs, Strings and Multiresolution , 2001 .

[18]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[19]  R. Tibshirani.  Adaptive piecewise polynomial estimation via trend filtering , 2013, 1304.2986.

[20]  Timothy A. Davis,et al.  Dynamic Supernodes in Sparse Cholesky Update/Downdate and Triangular Solves , 2009, TOMS.

[21]  Stephen P. Boyd,et al.  1 Trend Filtering , 2009, SIAM Rev..

[22]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[23]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[24]  Ryan J. Tibshirani,et al.  Fast and Flexible ADMM Algorithms for Trend Filtering , 2014, ArXiv.