Nonlinear principal component analysis by neural networks

Nonlinear principal component analysis (NLPCA) can be performed by a neural network model which nonlinearly generalizes the classical principal component analysis (PCA) method. The presence of local minima in the cost function renders the NLPCA somewhat unstable, as optimizations started from different initial parameters often converge to different minima. Regularization by adding weight penalty terms to the cost function is shown to improve the stability of the NLPCA. With the linear approach, there is a dichotomy between PCA and rotated PCA methods, as it is generally impossible to have a solution simultaneously(a) explaining maximum global variance of the data, and (b) approaching local data clusters. With the NLPCA, both objectives (a) and (b) can be attained together, thus the nonlinearity in NLPCA unifies the PCA and rotated PCA approaches. With a circular node at the network bottleneck, the NLPCA is able to extract periodic or wave modes. The Lorenz (1963) 3-component chaotic system and the monthly tropical Pacific sea surface temperatures (1950-1999) are used to illustrated the NLPCA approach.

[1]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[2]  William W. Hsieh,et al.  Applying Neural Network Models to Prediction and Data Analysis in Meteorology and Oceanography. , 1998 .

[3]  H. Storch,et al.  Statistical Analysis in Climate Research , 2000 .

[4]  Thomas M. Smith,et al.  Reconstruction of Historical Sea Surface Temperatures Using Empirical Orthogonal Functions , 1996 .

[5]  R. Preisendorfer,et al.  Principal Component Analysis in Meteorology and Oceanography , 1988 .

[6]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[7]  Min Zhong,et al.  El Niño, La Niña, and the Nonlinearity of Their Teleconnections , 1997 .

[8]  Adam H. Monahan,et al.  Nonlinear Principal Component Analysis by Neural Networks: Theory and Application to the Lorenz System , 2000 .

[9]  T. McAvoy,et al.  Nonlinear principal component analysis—Based on principal curves and neural networks , 1996 .

[10]  G. North.  Empirical Orthogonal Functions and Normal Modes , 1984 .

[11]  R. Tibshirani,et al.  Adaptive Principal Surfaces , 1994 .

[12]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[13]  L. M. Berliner,et al.  Statistics, Probability and Chaos , 1992 .

[14]  Michael Ghil,et al.  Statistics and Dynamics of Persistent Anomalies , 1987 .

[15]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[16]  Ferdinand Hergert,et al.  Improving model selection by nonconvergent methods , 1993, Neural Networks.

[17]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[18]  Adam H. Monahan,et al.  Nonlinear Principal Component Analysis: Tropical Indo–Pacific Sea Surface Temperature and Sea Level Pressure , 2001 .

[19]  I. Jolliffe.  Principal Component Analysis , 2002 .

[20]  A. Barnston,et al.  Classification, seasonality and persistence of low-frequency atmospheric circulation patterns , 1987 .

[21]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[22]  A. Raftery,et al.  Ice Floe Identification in Satellite Images Using Mathematical Morphology and Clustering about Principal Curves , 1992 .

[23]  Thomas M. Smith,et al.  Improved Global Sea Surface Temperature Analyses Using Optimum Interpolation , 1994 .

[24]  W. White,et al.  North Pacific thermocline variations on ENSO timescales , 1997 .

[25]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[26]  William W. Hsieh,et al.  Nonlinear canonical correlation analysis by neural networks , 2000, Neural Networks.

[27]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[28]  Roland Baddeley,et al.  Nonlinear principal components analysis of neuronal spike train data , 1997, Biological Cybernetics.

[29]  R. Miranda,et al.  Circular Nodes in Neural Networks , 1996, Neural Computation.

[30]  A. Barnston,et al.  Prediction of ENSO Episodes Using Canonical Correlation Analysis , 1992 .

[31]  W. Hsieh.  Nonlinear Canonical Correlation Analysis of the Tropical Pacific Climate Variability Using a Neural Network Approach , 2001 .

[32]  William H. Press,et al.  Numerical recipes in C , 2002 .

[33]  H. Kaiser.  The varimax criterion for analytic rotation in factor analysis , 1958 .

[34]  E. C. Malthouse,et al.  Limitations of nonlinear PCA as performed with generic neural networks , 1998, IEEE Trans. Neural Networks.

[35]  G. Burroughs,et al.  THE ROTATION OF PRINCIPAL COMPONENTS , 1961 .