Supervised Dimension Reduction of Intrinsically Low-Dimensional Data

High-dimensional data generated by a system with limited degrees of freedom are often constrained in low-dimensional manifolds in the original space. In this article, we investigate dimension-reduction methods for such intrinsically low-dimensional data through linear projections that preserve the manifold structure of the data. For intrinsically one-dimensional data, this implies projecting to a curve on the plane with as few intersections as possible. We are proposing a supervised projection pursuit method that can be regarded as an extension of the single-index model for nonparametric regression. We show results from a toy and two robotic applications.

[1]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[2]  M. Spivak A comprehensive introduction to differential geometry , 1979 .

[3]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[6]  Philip E. Gill,et al.  Practical optimization , 1981 .

[7]  B. Silverman,et al.  Algorithm AS 176: Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[8]  B. Silverman,et al.  Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[11]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[12]  W. Härdle,et al.  Optimal Smoothing in Single-index Models , 1993 .

[13]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[14]  Hiroshi Murase,et al.  Subspace methods for robot vision , 1996, IEEE Trans. Robotics Autom..

[15]  J. Príncipe,et al.  Information-Theoretic Learning Using Renyi's Quadratic Entropy , 1999 .

[16]  William M. Campbell,et al.  Mutual Information in Learning Feature Transformations , 2000, ICML.

[17]  Sebastian Thrun,et al.  Probabilistic Algorithms in Robotics , 2000, AI Mag..

[18]  Ben J. A. Kröse,et al.  Jijo-2: An Office Robot that Communicates and Learns , 2001, IEEE Intell. Syst..

[19]  Ben J. A. Kröse,et al.  Auxiliary particle filter robot localization from high-dimensional sensor observations , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[20]  Stergios I. Roumeliotis,et al.  Stochastic cloning: a generalized framework for processing relative state measurements , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[21]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[22]  Sebastian Thrun,et al.  Bayesian Landmark Learning for Mobile Robot Localization , 1998, Machine Learning.