Efficient Kernel Orthonormalized PLS for Remote Sensing Applications

This paper studies the performance and applicability of a novel kernel partial least squares (KPLS) algorithm for nonlinear feature extraction in the context of remote sensing applications. The so-called kernel orthonormalized PLS algorithm with reduced complexity (rKOPLS) has the following two core parts: (1) a kernel version of OPLS (called KOPLS) and (2) a sparse approximation for large-scale data sets, which ultimately leads to the rKOPLS algorithm. The method is theoretically analyzed in terms of computational burden and memory requirements and is tested in common remote sensing applications: multi- and hyperspectral image classification and biophysical parameter estimation problems. The proposed method largely outperforms the traditional (linear) PLS algorithm and demonstrates good capabilities in terms of expressive power of the extracted nonlinear features, accuracy, and scalability as compared to the standard KPLS.

[1]  Lorenzo Bruzzone,et al.  A Support Vector Domain Description Approach to Supervised Classification of Remote Sensing Images , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Terje Gobakken,et al.  Comparing regression methods in estimation of biophysical properties of forest stands from two different inventories using laser scanner data , 2005 .

[3]  Paul Geladi,et al.  Chemometrics in spectroscopyPart 2. Examples , 2004 .

[4]  Roman Rosipal,et al.  Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space , 2002, J. Mach. Learn. Res..

[5]  M. Kahru,et al.  Ocean Color Chlorophyll Algorithms for SEAWIFS , 1998 .

[6]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[7]  Gustavo Camps-Valls,et al.  Composite kernels for hyperspectral image classification , 2006, IEEE Geoscience and Remote Sensing Letters.

[8]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[9]  Amit Banerjee,et al.  A support vector method for anomaly detection in hyperspectral imagery , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[11]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[12]  C. Brody.  Linear Heteroencoders , 1999 .

[13]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[14]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[15]  Marco Diani,et al.  Retrieval of sea water optically active parameters from hyperspectral data by means of generalized radial basis function neural networks , 2001, IEEE Trans. Geosci. Remote. Sens..

[16]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[17]  I. Jolliffe.  Principal Component Analysis , 2002 .

[18]  S. Wold.  Nonlinear partial least squares modelling II. Spline inner relation , 1992 .

[19]  P. Geladi.  Notes on the history and nature of partial least squares (PLS) modelling , 1988 .

[20]  E. Martin,et al.  Non-linear projection to latent structures revisited: the quadratic PLS algorithm , 1999 .

[21]  Roman Rosipal,et al.  Overview and Recent Advances in Partial Least Squares , 2005, SLSFS.

[22]  J. Clevers.  The Derivation of a Simplified Reflectance Model for the Estimation of Leaf Area Index , 1988 .

[23]  Susan L. Ustin,et al.  Classification of contamination in salt marsh plants using hyperspectral reflectance , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[24]  David Riaño,et al.  Retrieval of Fresh Leaf Fuel Moisture Content Using Genetic Algorithm Partial Least Squares (GA-PLS) Modeling , 2007, IEEE Geoscience and Remote Sensing Letters.

[25]  Nicholas C. Coops,et al.  Prediction of eucalypt foliage nitrogen content from satellite-derived hyperspectral data , 2003, IEEE Trans. Geosci. Remote. Sens..

[26]  Kaare Brandt Petersen,et al.  Sparse Kernel Orthonormalized PLS for feature extraction in large data sets , 2006, NIPS.

[27]  José Luis Rojo-Álvarez,et al.  Robust support vector regression for biophysical variable estimation from remotely sensed images , 2006, IEEE Geoscience and Remote Sensing Letters.

[28]  Su-Yun Huang,et al.  Reduced Support Vector Machines: A Statistical Theory , 2007, IEEE Transactions on Neural Networks.

[29]  P. Geladi,et al.  Chemometrics and intelligent laboratory systems Plastic identification by remote sensing spectroscopic NIR imaging using kernel partial least squares ( KPLS ) , 2003 .

[30]  Jane R. Foster,et al.  Application of imaging spectroscopy to mapping canopy nitrogen in the forests of the central Appalachian Mountains using Hyperion and AVIRIS , 2003, IEEE Trans. Geosci. Remote. Sens..

[31]  S. Wold,et al.  Multivariate Data Analysis in Chemistry , 1984 .

[32]  I. E. Frank.  A nonlinear PLS model , 1990 .

[33]  José Luis Rojo-Álvarez,et al.  Kernel Methods in Bioengineering, Signal And Image Processing , 2007 .

[34]  Gustavo Camps-Valls,et al.  Feature extraction from remote sensing data using Kernel Orthonormalized PLS , 2007, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[35]  Paul Geladi,et al.  Chemometrics in spectroscopy : Part 2. Examples , 2004 .

[36]  Francesco Vuolo,et al.  Cost-effectiveness of Vegetation Biophysical Parameters Retrieval from Remote Sensing Data , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[37]  Karl J. Friston,et al.  Characterizing the Response of PET and fMRI Data Using Multivariate Linear Models , 1997, NeuroImage.

[38]  S. Wold,et al.  Nonlinear PLS modeling , 1989 .

[39]  Bart Nicolai,et al.  Kernel PLS regression on wavelet transformed NIR spectra for prediction of sugar content of apple , 2007 .

[40]  Serge Kokot,et al.  Simultaneous Determination of Three Organophosphorus Pesticides by Differential Pulse Stripping Voltammetry and Chemometrics , 2004 .

[41]  A. J. Morris,et al.  Non-linear projection to latent structures revisited (the neural network PLS algorithm) , 1999 .

[42]  H. Wold.  Path Models with Latent Variables: The NIPALS Approach , 1975 .

[43]  Nishimatsu Toyonori,et al.  Measurement of pesticide residues in food based on diffuse reflectance IR spectroscopy , 2002 .

[44]  J. Vandewalle,et al.  Primal space sparse kernel partial least squares regression for large scale problems , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[45]  Kristin P. Bennett,et al.  Sparse Kernel Partial Least Squares Regression , 2003, COLT.

[46]  M. G. O'shea,et al.  Role of chemometrics for at-field application of NIR spectroscopy to predict sugarcane clonal performance , 2007 .

[47]  F. Bookstein,et al.  Neurobehavioral effects of prenatal alcohol: Part II. Partial least squares analysis. , 1989, Neurotoxicology and teratology.

[48]  A-Xing Zhu,et al.  Prediction of Continental-Scale Evapotranspiration by Combining MODIS and AmeriFlux Data Through Support Vector Machine , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Lorenzo Bruzzone,et al.  Kernel-based methods for hyperspectral image classification , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Jacob A. Wegelin,et al.  A Survey of Partial Least Squares (PLS) Methods, with Emphasis on the Two-Block Case , 2000 .

[51]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..