Sliced Inverse Regression for Dimension Reduction

Abstract Modern advances in computing power have greatly widened scientists' scope in gathering and investigating information from many variables, information which might have been ignored in the past. Yet to effectively scan a large pool of variables is not an easy task, although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics. In this article, we propose a novel data-analytic tool, sliced inverse regression (SIR), for reducing the dimension of the input variable x without going through any parametric or nonparametric model-fitting process. This method explores the simplicity of the inverse view of regression; that is, instead of regressing the univariate output variable y against the multivariate x, we regress x against y. Forward regression and inverse regression are connected by a theorem that motivates this method. The theoretical properties of SIR are investigated under a model of the form, y = f(β 1 x, …, β K x, e), where the β k 's are the unknown...

[1]  McCollom Jh,et al.  The Discussion , 1897 .

[2]  H. Hotelling.  The most predictable criterion. , 1935 .

[3]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[4]  J. Hooper,et al.  Simultaneous Equations and Canonical Correlation Theory , 1959 .

[5]  C. L. Mallows.  Latent vectors of random symmetric matrices , 1961 .

[6]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[7]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[8]  D. Brillinger.  The identification of a particular nonlinear time series system , 1977 .

[9]  David E. Tyler.  Asymptotic Inference for Eigenvectors , 1981 .

[10]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[11]  P. Bickel,et al.  An Analysis of Transformations Revisited , 1981 .

[12]  T. Hassard,et al.  Applied Linear Regression , 2005 .

[13]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[14]  James Allen Fill,et al.  On Projection Pursuit Measures of Multivariate Location and Dispersion , 1984 .

[15]  D. Freedman,et al.  Asymptotics of Graphical Projection Pursuit , 1984 .

[16]  S. Portnoy.  Asymptotic behavior of M-estimators of p regression parameters when p , 1985 .

[17]  D. Hinkley,et al.  The Analysis of Transformed Data , 1984 .

[18]  David Ruppert,et al.  Power Transformations When Fitting Theoretical Models to Data , 1984 .

[19]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[20]  Guoying Li,et al.  Projection-Pursuit Approach to Robust Dispersion Matrices and Principal Components: Primary Theory and Monte Carlo , 1985 .

[21]  D. R. Cox,et al.  Discussion: Projection Pursuit , 1985 .

[22]  J. Friedman,et al.  Estimating Optimal Transformations for Multiple Regression and Correlation. , 1985 .

[23]  Werner A. Stahel,et al.  Discussion: Projection Pursuit , 1985 .

[24]  Nancy E. Heckman,et al.  Spline Smoothing in a Partly Linear Model , 1986 .

[25]  C. J. Stone,et al.  The Dimensionality Reduction Principle for Generalized Additive Models , 1986 .

[26]  Daniel B. Carr,et al.  Scatterplot matrix techniques for large N , 1986 .

[27]  A. A. Weiss,et al.  Semiparametric estimates of the relation between weather and electricity sales , 1986 .

[28]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[29]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[30]  Peter J. Huber,et al.  Experiences with Three-Dimensional Scatterplots , 1987 .

[31]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[32]  Robert A. Koyak,et al.  On Measuring Internal Dependence in a Set of Random Variables , 1987 .

[33]  William S. Cleveland,et al.  Research in Statistical Graphics , 1987 .

[34]  Ker-Chau Li,et al.  Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set , 1987 .

[35]  W. Loh,et al.  Tree-Structured Classification Via Generalized Discriminant Analysis: Rejoinder , 1988 .

[36]  Hung Chen,et al.  Convergence Rates for Parametric Components in a Partly Linear Model , 1988 .

[37]  W. Loh,et al.  Tree-Structured Classification via Generalized Discriminant Analysis. , 1988 .

[38]  W. Härdle,et al.  How Far are Automatically Chosen Regression Smoothing Parameters from their Optimum , 1988 .

[39]  P. Speckman.  Kernel smoothing in partial linear models , 1988 .

[40]  B. Yandell.  Spline smoothing and nonparametric regression , 1989 .

[41]  M. C. Jones,et al.  Spline Smoothing and Nonparametric Regression. , 1989 .

[42]  I. Johnstone,et al.  Projection-Based Approximation and a Duality with Kernel Methods , 1989 .

[43]  Ker-Chau Li,et al.  Regression Analysis Under Link Violation , 1989 .

[44]  Ker-Chau Li,et al.  Slicing Regression: A Link-Free Regression Method , 1991 .

[45]  K. Li.  Uncertainty analysis for mathematical models with SIR , 1992 .

[46]  Jack Cuzick,et al.  Semiparametric additive regression , 1992 .

[47]  Ker-Chau Li,et al.  On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma , 1992 .