Fused Lasso Additive Model

We consider the problem of predicting an outcome variable using p covariates that are measured on n independent observations, in a setting in which additive, flexible, and interpretable fits are desired. We propose the fused lasso additive model (FLAM), in which each additive function is estimated to be piecewise constant with a small number of adaptively chosen knots. FLAM is the solution to a convex optimization problem, for which a simple algorithm with guaranteed convergence to a global optimum is provided. FLAM is shown to be consistent in high dimensions, and an unbiased estimator of its degrees of freedom is proposed. We evaluate the performance of FLAM in a simulation study and on two datasets. Supplemental materials are available online, and the R package flam is available on CRAN.

[1]  Hadley Cantril,et al.  The pattern of human concerns , 1965 .

[2]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[3]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[4]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[5]  B. Efron.  How Biased is the Apparent Error Rate of a Prediction Rule , 1986 .

[6]  S. Geer,et al.  Locally adaptive regression splines , 1997 .

[7]  鳥居 泰彦,et al.  世界経済・社会統計 = World development indicators , 1998 .

[8]  P. Bühlmann,et al.  Boosting with the L2-loss: regression and classification , 2001 .

[9]  P. Tseng.  Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[10]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[11]  Hao Helen Zhang,et al.  Component selection and smoothing in smoothing spline analysis of variance models -- COSSO , 2003 .

[12]  P. Tseng,et al.  AMlet, RAMlet, and GAMlet: Automatic Nonlinear Fitting of Additive Models, Robust and Generalized, With Wavelets , 2004 .

[13]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[14]  Tong Zhang.  From ɛ-entropy to KL-entropy: Analysis of minimum information complexity density estimation , 2006, math/0702653.

[15]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[16]  Hao Helen Zhang,et al.  Component selection and smoothing in multivariate nonparametric regression , 2006, math/0702659.

[17]  P. Sebastiani,et al.  Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer , 2007, Nature Medicine.

[18]  J. Lafferty,et al.  Sparse additive models , 2007, 0711.4555.

[19]  Dennis B. Troup,et al.  NCBI GEO: mining tens of millions of expression profiles—database and tools update , 2006, Nucleic Acids Res..

[20]  Christophe Ambroise,et al.  Parsimonious additive models , 2007, Comput. Stat. Data Anal..

[21]  Peter Buhlmann,et al.  BOOSTING ALGORITHMS: REGULARIZATION, PREDICTION AND MODEL FITTING , 2007, 0804.2752.

[22]  Stephen P. Boyd,et al.  1 Trend Filtering , 2009, SIAM Rev..

[23]  Holger Hoefling.  A Path Algorithm for the Fused Lasso Signal Approximator , 2009, 0910.0526.

[24]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[25]  S. Geer,et al.  High-dimensional additive modeling , 2008, 0806.4115.

[26]  Jieping Ye,et al.  An efficient algorithm for a class of fused lasso problems , 2010, KDD.

[27]  Chuhsing Kate Hsiao,et al.  Identification of a Novel Biomarker, SEMA5A, for Non–Small Cell Lung Carcinoma in Nonsmoking Women , 2010, Cancer Epidemiology, Biomarkers & Prevention.

[28]  J. Horowitz,et al.  VARIABLE SELECTION IN NONPARAMETRIC ADDITIVE MODELS. , 2010, Annals of statistics.

[29]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[30]  Dietrich A. Stephan,et al.  Autism and Increased Paternal Age Related Changes in Global Levels of Gene Expression Regulation , 2011, PloS one.

[31]  Yufeng Liu,et al.  Linear or Nonlinear? Automatic Structure Discovery for Partially Linear Models , 2011, Journal of the American Statistical Association.

[32]  Robert Tibshirani,et al.  STANDARDIZATION AND THE GROUP LASSO PENALTY. , 2012, Statistica Sinica.

[33]  R. Tibshirani,et al.  Degrees of freedom in lasso problems , 2011, 1111.0653.

[34]  Stephen P. Boyd,et al.  A Mathematical Model for Interpretable Clinical Decision Support with Applications in Gynecology , 2012, PloS one.

[35]  Nicholas A. Johnson,et al.  A Dynamic Programming Algorithm for the Fused Lasso and L 0-Segmentation , 2013 .

[36]  Johannes Gehrke,et al.  Sparse Partially Linear Additive Models , 2014, ArXiv.

[37]  R. Tibshirani.  Adaptive piecewise polynomial estimation via trend filtering , 2013, 1304.2986.

[38]  Ryan J. Tibshirani,et al.  Fast and Flexible ADMM Algorithms for Trend Filtering , 2014, ArXiv.

[39]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[40]  Yan Li.  Sparse Additive Model using Symmetric Nonnegative Definite Smoothers , 2014, ArXiv.

[41]  秀俊 松井,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2014 .

[42]  S. Wood,et al.  Generalized additive models for large data sets , 2015 .

[43]  Jj Allaire,et al.  Web Application Framework for R , 2016 .