Manifold-constrained regressors in system identification

High-dimensional regression problems are becoming more and more common with emerging technologies. However, in many cases data are constrained to a low dimensional manifold. The information about the output is hence contained in a much lower dimensional space, which can be expressed by an intrinsic description. By first finding the intrinsic description, a low dimensional mapping can be found to give us a two step mapping from regressors to output. In this paper a methodology aimed at manifold-constrained identification problems is proposed. A supervised and a semi-supervised method are presented, where the later makes use of given regressor data lacking associated output values for learning the manifold. As it turns out, the presented methods also carry some interesting properties also when no dimensional reduction is performed.

[1]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[2]  Bernhard Schölkopf,et al.  Semi-Supervised Learning (Adaptive Computation and Machine Learning) , 2006 .

[3]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[4]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[5]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[6]  Xilin Shen,et al.  Analysis of Event-Related fMRI Data Using Diffusion Maps , 2005, IPMI.

[7]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[8]  Xin Yang,et al.  Semi-supervised nonlinear dimensionality reduction , 2006, ICML.

[9]  Stan Z. Li,et al.  Manifold Learning and Applications in Recognition , 2005 .

[10]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[11]  Jie Tian,et al.  Functional Feature Embedded Space Mapping of fMRI data , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Volker Mehrmann,et al.  Differential-Algebraic Equations: Analysis and Numerical Solution , 2006 .

[13]  Olivier D. Faugeras,et al.  Nonlinear dimension reduction of fMRI data: the Laplacian embedding approach , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[14]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[15]  D. Lindgren.  Projection Techniques for Classification and Identification , 2005 .

[16]  Ker-Chau Li,et al.  On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma , 1992 .

[17]  Hongtao Lu,et al.  Supervised LLE in ICA Space for Facial Expression Recognition , 2005, 2005 International Conference on Neural Networks and Brain.

[18]  Michael I. Jordan,et al.  Dimensionality Reduction for Supervised Learning with Reproducing Kernel Hilbert Spaces , 2004, J. Mach. Learn. Res..

[19]  L. Ljung,et al.  USING MANIFOLD LEARNING FOR NONLINEAR SYSTEM IDENTIFICATION , 2007 .