Functional inverse regression and reproducing kernel Hilbert space

Functional Inverse Regression and Reproducing Kernel Hilbert Space. (August 2005) Haobo Ren, B.S., Peking University; M.S., Peking University Chair of Advisory Committee: Dr. Tailen Hsing The basic philosophy of Functional Data Analysis (FDA) is to think of the observed data functions as elements of a possibly infinite-dimensional fu nction space. Most of the current research topics on FDA focus on advancing theoretical t ools and extending existing multivariate techniques to accommodate the infinite-dimen sional nature of data. This dissertation reports contributions on both fronts, where a uni fying inverse regression theory for both the multivariate setting (Li 1991) and functional d ta from a Reproducing Kernel Hilbert Space (RKHS) prospective is developed. We proposed a functional multiple-index model which models a real response variable as a function of a few predictor variables called indice s. These indices are random elements of the Hilbert space spanned by a second order stoch a ic process and they constitute the so-called Effective Dimensional Reduction Spac e (EDRS). To conduct inference on the EDRS, we discovered a fundamental result which reveals the geometrical association between the EDRS and the RKHS of the process. Two inverse re gr ssion procedures, a “slicing” approach and a kernel approach, were introduced to estimate the counterpart of the EDRS in the RKHS. Further the estimate of the EDRS was achieve d via the transformation from the RKHS to the original Hilbert space. To constru ct an asymptotic theory, we introduced an isometric mapping from the empirical RKHS to th e theoretical RKHS, which can be used to measure the distance between the estimator and the target. Some general

[1]  J. Polzehl,et al.  Structure adaptive approach for dimension reduction , 2001 .

[2]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[3]  A. Juditsky,et al.  Direct estimation of the index coefficient in a single-index model , 2001 .

[4]  Kok-Kwang Phoon,et al.  Convergence study of the truncated Karhunen–Loeve expansion for simulation of stochastic processes , 2001 .

[5]  H. Müller,et al.  Functional Data Analysis for Sparse Longitudinal Data , 2005 .

[6]  Frédéric Ferraty,et al.  The Functional Nonparametric Model and Application to Spectrometric Data , 2002, Comput. Stat..

[7]  Ursula Gather,et al.  A note On outlier sensitivity of Sliced Inverse Regression , 2002 .

[8]  Jianqing Fan,et al.  Two‐step estimation of functional linear models with applications to longitudinal data , 1999 .

[9]  K. Fang,et al.  Asymptotics for kernel estimate of sliced inverse regression , 1996 .

[10]  H. Vincent Poor,et al.  An RKHS approach to robust L2 estimation and signal detection , 1990, IEEE Trans. Inf. Theory.

[11]  L. Ferré Determining the Dimension in Sliced Inverse Regression and Related Methods , 1998 .

[12]  Constantinos Goutis,et al.  Second‐derivative functional regression with applications to near infra‐red spectroscopy , 1998 .

[13]  Lixing Zhu,et al.  Asymptotics of sliced inverse regression , 1995 .

[14]  A. Tsybakov,et al.  Sliced Inverse Regression for Dimension Reduction - Comment , 1991 .

[15]  Jeanne Fine Asymptotic study of canonical correlation analysis: from matrix and analytic approach to operator and tensor approach , 2003 .

[16]  Gilbert Saporta,et al.  Clusterwise PLS regression on a stochastic process , 2002, Comput. Stat. Data Anal..

[17]  J. Saracco,et al.  AN ASYMPTOTIC THEORY FOR SIRα METHOD , 2003 .

[18]  Algirdas Laukaitis,et al.  Functional Data Analysis of Payment Systems , 2002 .

[19]  John A. Rice,et al.  FUNCTIONAL AND LONGITUDINAL DATA ANALYSIS: PERSPECTIVES ON SMOOTHING , 2004 .

[20]  Philippe Besse,et al.  Simultaneous non-parametric regressions of unbalanced longitudinal data , 1997 .

[21]  Jane-Ling Wang,et al.  Functional quasi‐likelihood regression models with smooth random effects , 2003 .

[22]  Joel L. Horowitz,et al.  Methodology and convergence rates for functional linear regression , 2007, 0708.0466.

[23]  H. Weinert Reproducing kernel Hilbert spaces: Applications in statistical signal processing , 1982 .

[24]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[25]  B. Silverman,et al.  Canonical correlation analysis when the data are curves. , 1993 .

[26]  B. Silverman,et al.  Smoothed functional principal components analysis by choice of norm , 1996 .

[27]  Tailen Hsing,et al.  Nearest neighbor inverse regression , 1999 .

[28]  A. M. Aguilera,et al.  Principal component estimation of functional logistic regression: discussion of two different approaches , 2004 .

[29]  D. Ruppert,et al.  Penalized Spline Estimation for Partially Linear Single-Index Models , 2002 .

[30]  Henry W. Altland,et al.  Applied Functional Data Analysis , 2003, Technometrics.

[31]  Kerby Shedden,et al.  Dimension Reduction for Multivariate Response Data , 2003 .

[32]  Jérôme Saracco,et al.  An asymptotic theory for sliced inverse regression , 1997 .

[33]  H. Müller,et al.  Functional Convex Averaging and Synchronization for Time-Warped Random Curves , 2004 .

[34]  M. Driscoll The reproducing kernel Hilbert space structure of the sample paths of a Gaussian process , 1973 .

[35]  Gareth M. James,et al.  Functional Adaptive Model Estimation , 2005 .

[36]  H. Müller,et al.  Shrinkage Estimation for Functional Principal Component Scores with Application to the Population Kinetics of Plasma Folate , 2003, Biometrics.

[37]  Michel Verleysen,et al.  Representation of functional data in neural networks , 2005, Neurocomputing.

[38]  Chun-Houh Chen,et al.  CAN SIR BE AS POPULAR AS MULTIPLE LINEAR REGRESSION , 2003 .

[39]  H. Tong,et al.  Article: 2 , 2002, European Financial Services Law.

[40]  H. Cardot,et al.  Estimation in generalized linear models for functional data via penalized likelihood , 2005 .

[41]  Ker-Chau Li,et al.  On almost Linearity of Low Dimensional Projections from High Dimensional Data , 1993 .

[42]  Chong Gu Smoothing Spline Anova Models , 2002 .

[43]  T. Gasser,et al.  Statistical Tools to Analyze Data Representing a Sample of Curves , 1992 .

[44]  R. Cook,et al.  Estimating the structural dimension of regressions via parametric inverse regression , 2001 .

[45]  P. Sarda,et al.  SPLINE ESTIMATORS FOR THE FUNCTIONAL LINEAR MODEL , 2003 .

[46]  Thomas Kailath,et al.  RKHS approach to detection and estimation problems-I: Deterministic signals in Gaussian noise , 1971, IEEE Trans. Inf. Theory.

[47]  Colin O. Wu,et al.  Nonparametric Mixed Effects Models for Unequally Sampled Noisy Curves , 2001, Biometrics.

[48]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[49]  J. Dauxois,et al.  Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference , 1982 .

[50]  T. Gasser,et al.  Searching for Structure in Curve Samples , 1995 .

[51]  J. Ramsay,et al.  Some Tools for Functional Data Analysis , 1991 .

[52]  H. Cardot Nonparametric estimation of smoothed principal components analysis of sampled noisy functions , 2000 .

[53]  Raymond J. Carroll,et al.  Measurement Error Regression with Unknown Link: Dimension Reduction and Data Visualization , 1992 .

[54]  Prasad A. Naik,et al.  Constrained Inverse Regression for Incorporating Prior Information , 2005 .

[55]  Jane-Ling Wang,et al.  A FUNCTIONAL MULTIPLICATIVE EFFECTS MODEL FOR LONGITUDINAL DATA, WITH APPLICATION TO REPRODUCTIVE HISTORIES OF FEMALE MEDFLIES. , 2003, Statistica Sinica.

[56]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[57]  B. Silverman,et al.  Estimating the mean and covariance structure nonparametrically when the data are curves , 1991 .

[58]  Frédéric Ferraty,et al.  Nonparametric models for functional data, with application in regression, time series prediction and curve discrimination , 2004 .

[59]  W. Fung,et al.  DIMENSION REDUCTION BASED ON CANONICAL CORRELATION , 2002 .

[60]  André Mas,et al.  Testing hypotheses in the functional linear model , 2003 .

[61]  R. Dennis Cook,et al.  K-Means Inverse Regression , 2004, Technometrics.

[62]  Pascal Sarda,et al.  Conditional quantiles with functional covariates: an application to ozone pollution forecasting , 2004 .

[63]  R. Cook Regression Graphics , 1994 .

[64]  J. Simonoff Smoothing Methods in Statistics , 1998 .

[65]  M. Pourahmadi,et al.  Nonparametric estimation of large covariance matrices of longitudinal data , 2003 .

[66]  E. Parzen An Approach to Time Series Analysis , 1961 .

[67]  G. Wahba Spline models for observational data , 1990 .

[68]  Wensheng Guo,et al.  Functional mixed effects models , 2012, Biometrics.

[69]  T. Hsing,et al.  Canonical correlation for stochastic processes , 2008 .

[70]  Anestis Antoniadis,et al.  Dimension reduction in functional regression with applications , 2006, Comput. Stat. Data Anal..

[71]  B. Mandelbrot,et al.  Fractional Brownian Motions, Fractional Noises and Applications , 1968 .

[72]  Marcia M. A. Schafgans,et al.  A derivative based estimator for semiparametric index models , 2003 .

[73]  Claudia Becker,et al.  Sliced Inverse Regression for High-dimensional Time Series , 2003 .

[74]  Ker-Chau Li,et al.  Nonlinear confounding in high-dimensional regression , 1997 .

[75]  L. Ferré,et al.  Functional sliced inverse regression analysis , 2003 .

[76]  J. Ramsay,et al.  The historical functional linear model , 2003 .

[77]  Kai-Tai Fang,et al.  The Classification Tree Combined with SIR and Its Applications to Classification of Mass Spectra , 2003, Journal of Data Science.

[78]  P. Sarda,et al.  Functional linear model , 1999 .

[79]  B. Yandell Spline smoothing and nonparametric regression , 1989 .

[80]  H. Müller,et al.  Methods of canonical analysis for functional data , 2004 .

[81]  Ker-Chau Li,et al.  Slicing Regression: A Link-Free Regression Method , 1991 .

[82]  A. Bowman,et al.  Applied smoothing techniques for data analysis : the kernel approach with S-plus illustrations , 1999 .

[83]  Jane-Ling Wang,et al.  Functional canonical analysis for square integrable stochastic processes , 2003 .

[84]  Ruth M. Pfeiffer,et al.  Graphical Methods for Class Prediction Using Dimension Reduction Techniques on DNA Microarray Data , 2003, Bioinform..

[85]  Gareth M. James Generalized linear models with functional predictors , 2002 .

[86]  Catherine A. Sugar,et al.  Principal component models for sparse functional data , 1999 .

[87]  E. Parzen Regression Analysis of Continuous Parameter Time Series , 1961 .

[88]  James R. Schott,et al.  Determining the Dimensionality in Sliced Inverse Regression , 1994 .

[89]  Prasad A. Naik,et al.  Partial least squares estimator for single‐index models , 2000 .

[90]  Ursula Gather,et al.  A Robustified Version of Sliced Inverse Regression , 2001 .

[91]  A. Cuevas,et al.  Linear functional regression: The case of fixed design and functional response , 2002 .

[92]  H. Muller,et al.  Generalized functional linear models , 2005, math/0505638.

[93]  P J Diggle,et al.  Nonparametric estimation of covariance structure in longitudinal data. , 1998, Biometrics.

[94]  T. Tony Cai,et al.  Prediction in functional linear regression , 2006 .

[95]  Gilbert Saporta,et al.  PLS regression on a stochastic process , 2001, Comput. Stat. Data Anal..

[96]  A. Goia Selection Model in Functional Linear Regression Models for Scalar Response , 2003 .

[97]  Dimension Choice for Sliced Inverse Regression Based on Ranks , .

[98]  Denis Bosq,et al.  Modelization, Nonparametric Estimation and Prediction for Continuous Time Processes , 1991 .