The Group-Lasso for generalized linear models: uniqueness of solutions and efficient algorithms

The Group-Lasso method for finding important explanatory factors suffers from the potential non-uniqueness of solutions and also from high computational costs. We formulate conditions for the uniqueness of Group-Lasso solutions which lead to an easily implementable test procedure that allows us to identify all potentially active groups. These results are used to derive an efficient algorithm that can deal with input dimensions in the millions and can approximate the solution path efficiently. The derived methods are applied to large-scale learning problems where they exhibit excellent performance and where the testing procedure helps to avoid misinterpretations of the solutions.

[1]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[2]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[3]  G. Crooks,et al.  WebLogo: A sequence logo generator, Genome Research, , 2004 .

[4]  S. Sathiya Keerthi,et al.  A simple and efficient algorithm for gene selection using sparse logistic regression , 2003, Bioinform..

[5]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[6]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[7]  M. R. Osborne,et al.  On the LASSO and its Dual , 2000 .

[8]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[9]  Martin J. Wainwright,et al.  A new class of upper bounds on the log partition function , 2002, IEEE Transactions on Information Theory.

[10]  Christopher B. Burge,et al.  Maximum entropy modeling of short sequence motifs with applications to RNA splicing signals , 2003, RECOMB '03.

[11]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[12]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[13]  Peter Bühlmann,et al.  Penalized likelihood for sparse contingency tables with an application to full-length cDNA libraries , 2007, BMC Bioinformatics.

[14]  R. W. Wedderburn,et al.  On the existence and uniqueness of the maximum likelihood estimates for certain generalized linear models , 1976 .

[15]  Jesper Tegnér,et al.  Consistent Feature Selection for Pattern Recognition in Polynomial Time , 2007, J. Mach. Learn. Res..

[16]  G. Crooks,et al.  WebLogo: a sequence logo generator. , 2004, Genome research.