Penalized partially linear models using sparse representations with an application to fMRI time series

In this paper, we consider modeling the nonparametric component in partially linear models (PLMs) using linear sparse representations, e.g., wavelet expansions. Two types of representations are investigated, namely, orthogonal bases (complete) and redundant overcomplete expansions. For bases, we introduce a regularized estimator of the nonparametric part. The important contribution here is that the nonparametric part can be parsimoniously estimated by choosing an appropriate penalty function for which the hard and soft thresholding estimators are special cases. This allows us to represent in an effective manner a broad class of signals, including stationary and/or nonstationary signals and avoids excessive bias in estimating the parametric component. We also give a fast estimation algorithm. The method is then generalized to handle the case of overcomplete representations. A large-scale simulation study is conducted to illustrate the finite sample properties of the estimator. The estimator is finally applied to real neurophysiological functional magnetic resonance imaging (MRI) data sets that are suspected to contain both smooth and transient drift features.

[1]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[2]  Pierre Vandergheynst,et al.  A simple test to check the optimality of a sparse signal approximation , 2006, Signal Process..

[3]  M Durban,et al.  Approximate Standard Errors in Semiparametric Models , 1999, Biometrics.

[4]  R. L. Eubank,et al.  Testing Goodness-of-Fit in Regression Via Order Selection Criteria , 1992 .

[5]  Wolfgang Härdle,et al.  BOOTSTRAP INFERENCE IN SEMIPARAMETRIC GENERALIZED ADDITIVE MODELS , 2004, Econometric Theory.

[6]  Stefano Alliney,et al.  An algorithm for the minimization of mixed l1 and l2 norms with application to Bayesian estimation , 1994, IEEE Trans. Signal Process..

[7]  E. Bullmore,et al.  Wavelet-Generalized Least Squares: A New BLU Estimator of Linear Regression Models with 1/f Errors , 2002, NeuroImage.

[8]  W. Härdle,et al.  Kernel regression smoothing of time series , 1992 .

[9]  R. Tibshirani,et al.  Linear Smoothers and Additive Models , 1989 .

[10]  T. Severini,et al.  Quasi-Likelihood Estimation in Semiparametric Models , 1994 .

[11]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[12]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[13]  François G. Meyer.  Wavelet based estimation of a semi parametric generalized linear model of fMRI time-series , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[14]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[15]  Nancy E. Heckman,et al.  Spline Smoothing in a Partly Linear Model , 1986 .

[16]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[17]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[18]  G. Wahba.  Smoothing noisy data with spline functions , 1975 .

[19]  P. K. Bhattacharya,et al.  Semiparametric inference in a partial linear model , 1997 .

[20]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[21]  Adhemar Bultheel,et al.  Generalized cross validation for wavelet thresholding , 1997, Signal Process..

[22]  Randall Eubank,et al.  Convergence rates for estimation in certain partially linear models , 1989 .

[23]  Jianqing Fan,et al.  Regularization of Wavelet Approximations , 2001 .

[24]  Wolfgang Härdle,et al.  Partially Linear Models , 2000 .

[25]  Xiao-Wen Chang,et al.  Wavelet estimation of partially linear models , 2004, Comput. Stat. Data Anal..

[26]  Hung Chen,et al.  Convergence Rates for Parametric Components in a Partly Linear Model , 1988 .

[27]  Jianqing Fan,et al.  Generalized Partially Linear Single-Index Models , 1997 .

[28]  François G. Meyer.  Wavelet-based estimation of a semiparametric generalized linear model of fMRI time-series , 2003, IEEE Transactions on Medical Imaging.

[29]  Henry Horng-Shing Lu,et al.  Generalized Cross-Validation for Wavelet Shrinkage in Nonparametric Mixed Effects Models , 2003 .

[30]  Paul L. Speckman,et al.  Curve fitting by polynomial-trigonometric regression , 1990 .

[31]  J. Rice.  Convergence rates for partially splined models , 1986 .

[32]  P. Speckman.  Kernel smoothing in partial linear models , 1988 .

[33]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[34]  J. Schmee.  An Introduction to Multivariate Statistical Analysis , 1986 .

[35]  P. Tseng,et al.  Block Coordinate Relaxation Methods for Nonparametric Wavelet Denoising , 2000 .

[36]  Mila Nikolova,et al.  Local Strong Homogeneity of a Regularized Estimator , 2000, SIAM J. Appl. Math..

[37]  Iwao Kanno,et al.  Activation detection in functional MRI using subspace modeling and maximum likelihood estimation , 1999, IEEE Transactions on Medical Imaging.

[38]  Michael G. Schimek,et al.  Estimation in partially linear models , 1998 .

[39]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[40]  P. Tseng,et al.  AMlet, RAMlet, and GAMlet: Automatic Nonlinear Fitting of Additive Models, Robust and Generalized, With Wavelets , 2004 .

[41]  Clifford H. Spiegelman,et al.  Testing the Goodness of Fit of a Linear Model via Nonparametric Regression Techniques , 1990 .