Developments and Applications of Nonlinear Principal Component Analysis – a Review

Although linear principal component analysis (PCA) originates from the work of Sylvester [67] and Pearson [51], the development of nonlinear counterparts has only received attention from the 1980s. Work on nonlinear PCA, or NLPCA, can be divided into the utilization of autoassociative neural networks, principal curves and manifolds, kernel approaches or the combination of these approaches. This article reviews existing algorithmic work, shows how a given data set can be examined to determine whether a conceptually more demanding NLPCA model is required and lists developments of NLPCA algorithms. Finally, the paper outlines problem areas and challenges that require future work to mature the NLPCA research field.

[1]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[2]  R. Fisher,et al.  STUDIES IN CROP VARIATION , 2009 .

[3]  H. Hotelling Analysis of a complex of statistical variables into principal components. , 1933 .

[4]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[5]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[6]  S. Wold Cross-Validatory Estimation of the Number of Components in Factor and Principal Components Models , 1978 .

[7]  Edmund R. Malinowski,et al.  Factor Analysis in Chemistry , 1980 .

[8]  J. E. Jackson Principal Components and Factor Analysis: Part III - What is Factor Analysis? , 1981 .

[9]  A. D. Gordon,et al.  Interpreting multivariate data , 1982 .

[10]  Jan de Leeuw,et al.  Nonlinear Principal Component Analysis , 1982 .

[11]  Vic Barnett,et al.  Interpreting multivariate data , 1982 .

[12]  Gene H. Golub,et al.  Matrix computations , 1983 .

[13]  Wojtek J. Krzanowski,et al.  Cross-validatory choice in principal component analysis; some sampling results , 1983 .

[14]  T. Hastie Principal Curves and Surfaces , 1984 .

[15]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[16]  Thomas Kailath,et al.  Detection of signals by information theoretic criteria , 1985, IEEE Trans. Acoust. Speech Signal Process..

[17]  Torsten Söderström,et al.  Model-structure selection by cross-validation , 1986 .

[18]  K. Esbensen,et al.  Strategy of multivariate image analysis (MIA) , 1989 .

[19]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[20]  M. Kramer Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[21]  R. Tibshirani Principal curves revisited , 1992 .

[22]  J. Edward Jackson,et al.  A User's Guide to Principal Components. , 1991 .

[23]  Ivan Dvořák,et al.  Singular-value decomposition in attractor reconstruction: pitfalls and precautions , 1992 .

[24]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[25]  Ricardo D. Fierro,et al.  The Total Least Squares Problem: Computational Aspects and Analysis (S. Van Huffel and J. Vandewalle) , 1993, SIAM Rev..

[26]  Lalit M. Patnaik,et al.  Genetic algorithms: a survey , 1994, Computer.

[27]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[28]  J. Macgregor,et al.  Monitoring batch processes using multiway principal component analysis , 1994 .

[29]  M. Verhaegen,et al.  Identifying MIMO Wiener systems using subspace model identification methods , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[30]  Shufeng Tan,et al.  Reducing data dimensionality through optimizing neural network inputs , 1995 .

[31]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[32]  M. Verhaegen,et al.  Identifying MIMO Hammerstein systems in the context of subspace model identification methods , 1996 .

[33]  W. Stuetzle,et al.  Extremal properties of principal curves in the plane , 1996 .

[34]  Werner Stuetzle,et al.  Geometric Properties of Principal Curves in the Plane , 1996 .

[35]  T. McAvoy,et al.  Nonlinear principal component analysis—Based on principal curves and neural networks , 1996 .

[36]  M. Verhaegen,et al.  Recursive subspace identification of linear and nonlinear Wiener type models , 1998, Proceedings of the 1998 IEEE International Conference on Control Applications (Cat. No.98CH36104).

[37]  Adam Krzyzak,et al.  A Polygonal Line Algorithm for Constructing Principal Curves , 1998, NIPS.

[38]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[39]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[40]  M. Niranjan,et al.  SUBSPACE MODELS FOR SPEECH TRANSITIONS USING PRINCIPAL , 1998 .

[41]  Joydeep Ghosh,et al.  Principal curve classifier-a nonlinear approach to pattern classification , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[42]  B. Bakshi Multiscale PCA with application to multivariate statistical process monitoring , 1998 .

[43]  A. J. Morris,et al.  Non-linear principal components analysis for process fault detection , 1998 .

[44]  Mahesan Niranjan,et al.  Parametric subspace modeling of speech transitions , 1999, Speech Commun..

[45]  Helge Ritter,et al.  Principal Curve Sonification , 2000 .

[46]  Adam Krzyzak,et al.  Learning and Design of Principal Curves , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  S. Kulkarni,et al.  Principal curves with bounded turn , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[48]  Michel Verhaegen,et al.  Recursive subspace identification of linear and non-linear Wiener state-space models , 2000, Autom..

[49]  M. G. Bader,et al.  Design and applications , 2000 .

[50]  B. Kégl,et al.  Principal curves: learning, design, and applications , 2000 .

[51]  Joydeep Ghosh,et al.  A Unified Model for Probabilistic Principal Surfaces , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Michael J. Piovoso,et al.  On unifying multiblock analysis with application to decentralized process monitoring , 2001 .

[53]  R. C. Williamson,et al.  Regularized principal manifolds , 2001 .

[54]  P. Delicado Another Look at Principal Curves and Surfaces , 2001 .

[55]  Allan D. Jepson,et al.  Sparse PCA: Extracting Multi-scale Structure from Data , 2001, ICCV.

[56]  I. Jolliffe Principal Component Analysis , 2002 .

[57]  Theodore B. Trafalis,et al.  Data mining techniques for improved WSR-88D rainfall estimation , 2002 .

[58]  Ben J. A. Kröse,et al.  A k-segments algorithm for finding principal curves , 2002, Pattern Recognit. Lett..

[59]  K. Kim,et al.  Face recognition using kernel principal component analysis , 2002, IEEE Signal Process. Lett..

[60]  Truman R. Brown,et al.  NMR Spectral Quantitation by Principal Component Analysis. , 2002 .

[61]  Jianming Yang,et al.  Face recognition using improved principal component analysis , 2003, MHS2003. Proceedings of 2003 International Symposium on Micromechatronics and Human Science (IEEE Cat. No.03TH8717).

[62]  Qian Du,et al.  Linear mixture analysis-based compression for hyperspectral image analysis , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Thierry Denoeux,et al.  Principal component analysis of fuzzy data using autoassociative neural networks , 2004, IEEE Transactions on Fuzzy Systems.

[64]  In-Beum Lee,et al.  Nonlinear dynamic process monitoring based on dynamic kernel PCA , 2004 .

[65]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[66]  Dewang Chen,et al.  Freeway traffic stream modeling based on principal curves and its analysis , 2004, IEEE Transactions on Intelligent Transportation Systems.

[67]  Jose L. Martinez-Morales,et al.  Geometric data fitting , 2004 .

[68]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[69]  J. Edward Jackson,et al.  A User's Guide to Principal Components: Jackson/User's Guide to Principal Components , 2004 .

[70]  Ivor W. Tsang,et al.  The pre-image problem in kernel methods , 2003, IEEE Transactions on Neural Networks.

[71]  Feature Extraction Techniques for the Analysis of Spectral Polarization Profiles , 2005, astro-ph/0410565.

[72]  D. S. Bernstein,et al.  Subspace identification for non-linear systems with measured-input non-linearities , 2005 .

[73]  George W. Irwin,et al.  Introduction of a nonlinearity measure for principal component models , 2005, Comput. Chem. Eng..

[74]  Enrique Baeyens,et al.  Subspace-based Identification Algorithms for Hammerstein and Wiener Models , 2005, Eur. J. Control.

[75]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[76]  ChangKyoo Yoo,et al.  Nonlinear multivariate filtering and bioprocess monitoring for supervising nonlinear biological processes , 2006 .

[77]  Rama Chellappa,et al.  Principal components null space analysis for image and video classification , 2006, IEEE Transactions on Image Processing.

[78]  David Suter,et al.  An Analysis of Linear Subspace Approaches for Computer Vision and Pattern Recognition , 2006, International Journal of Computer Vision.

[79]  Xiaolei Zou,et al.  Application of Principal Component Analysis to CHAMP Radio Occultation Data for Quality Control and a Diagnostic Study , 2006 .

[80]  Sherif Yehia,et al.  PCA-Based algorithm for unsupervised bridge crack detection , 2006, Adv. Eng. Softw..

[81]  Dewang Chen,et al.  Constraint K-Segment Principal Curves , 2006, ICIC.

[82]  Ramalingam Shanmugam,et al.  At a crossroad of data envelopment and principal component analyses , 2007 .

[83]  Hyun-Woo Cho Nonlinear feature extraction and classification of multivariate data in kernel feature space , 2007, Expert Syst. Appl..

[84]  T. Hastie,et al.  Principal Curves , 2007 .

[85]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[86]  D.,et al.  ICE FLOE IDENTIFICATION IN SATELLITE IMAGES USING MATHEMATICAL MORPHOLOGY AND CLUSTERING ABOUT PRINCIPAL CURVES , .