Hierarchical Manifold Learning With Applications to Supervised Classification for High-Resolution Remotely Sensed Images

Manifold learning is one of the representative nonlinear dimensionality reduction techniques and has had many successful applications in the fields of information processing, especially pattern classification, and computer vision. However, when it is used for supervised classification, in particular for hierarchical classification, the result is still unsatisfactory. To address this issue, a novel supervised approach, namely hierarchical manifold learning (HML) is proposed. HML takes into account both the between-class label information and the within-class local structural information of the training sets simultaneously to guide the dimension reduction process for classification purpose. In this process, we extract sharing features to represent the parent manifold's information, and better solve the out-of-sample problem of manifold learning by using the generalized regression neural network at considerably lower computational cost, thereby making the proposed HML more suitable for supervised classification. Experimental results demonstrate the feasibility and effectiveness of our proposed algorithm.

[1]  Peter Tiño,et al.  Multiple Manifolds Learning Framework Based on Hierarchical Mixture Density Model , 2008, ECML/PKDD.

[2]  Donald F. Specht,et al.  The general regression neural network - Rediscovered , 1993, Neural Networks.

[3]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Joydeep Ghosh,et al.  Improved Nonlinear Manifold Learning for Land Cover Classification via Intelligent Landmark Selection , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[5]  Antonio Torralba,et al.  Sharing Visual Features for Multiclass and Multiview Object Detection , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  Zhi-Hua Zhou,et al.  Supervised nonlinear dimensionality reduction for visualization and classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[10]  I. T. Jolliffe,et al.  Generalizations and Adaptations of Principal Component Analysis , 1986 .

[11]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[12]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[13]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[14]  Honggang Zhang,et al.  Comments on "Globally Maximizing, Locally Minimizing: Unsupervised Discriminant Projection with Application to Face and Palm Biometrics" , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[16]  Thomas L. Ainsworth,et al.  Bathymetric Retrieval From Hyperspectral Imagery Using Manifold Coordinate Representations , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[18]  Mikhail Belkin,et al.  Using manifold structure for partially labelled classification , 2002, NIPS 2002.

[19]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Qijun Zhao,et al.  Facial expression recognition on multiple manifolds , 2011, Pattern Recognit..

[21]  Hong Huo,et al.  Supervised classification of multispectral remote sensing images based on the nearest reduced convex hull approach , 2009 .

[22]  Thomas L. Ainsworth,et al.  Exploiting manifold geometry in hyperspectral imagery , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[24]  Thomas L. Ainsworth,et al.  Improved Manifold Coordinate Representations of Large-Scale Hyperspectral Scenes , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  H. Deutsch Principle Component Analysis , 2004 .

[27]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[28]  Yen-Wei Chen,et al.  Classification of High-Resolution Satellite Images Using Supervised Locality Preserving Projections , 2008, KES.

[29]  D. Flanders,et al.  Preliminary evaluation of eCognition object-based software for cut block delineation and feature extraction , 2003 .

[30]  Christopher M. Bishop,et al.  A Hierarchical Latent Variable Model for Data Visualization , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[32]  Joydeep Ghosh,et al.  Multiresolution manifold learning for classification of hyperspectral data , 2007, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[33]  Stan Z. Li,et al.  Manifold Learning and Applications in Recognition , 2005 .

[34]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, CVPR.

[35]  Kilian Q. Weinberger,et al.  An Introduction to Nonlinear Dimensionality Reduction by Maximum Variance Unfolding , 2006, AAAI.

[36]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Xi Chen,et al.  Graph-Based Feature Selection for Object-Oriented Classification in VHR Airborne Imagery , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[39]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[40]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[41]  H. Sebastian Seung,et al.  The Manifold Ways of Perception , 2000, Science.

[42]  Carlo Vercellis,et al.  Kernel ridge regression for out-of-sample mapping in supervised manifold learning , 2012, Expert Syst. Appl..

[43]  Gunnar Rätsch,et al.  Constructing Descriptive and Discriminative Nonlinear Features: Rayleigh Coefficients in Kernel Feature Spaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[45]  María José del Jesús,et al.  Genetic feature selection in a fuzzy rule-based classification system learning process for high-dimensional problems , 2001, Inf. Sci..

[46]  Matti Pietikäinen,et al.  Classification of handwritten digits using supervised locally linear embedding algorithm and support vector machine , 2003, ESANN.

[47]  Adam Krzyzak,et al.  Piecewise Linear Skeletonization Using Principal Curves , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[49]  Wenbin Chen,et al.  Manifold Learning for Image Denoising , 2005, The Fifth International Conference on Computer and Information Technology (CIT'05).

[50]  Xiangyang Xue,et al.  Metric learning by discriminant neighborhood embedding , 2008, Pattern Recognit..

[51]  Chi Hau Chen,et al.  Statistical pattern recognition in remote sensing , 2008, Pattern Recognit..

[52]  Jian Yang,et al.  Globally Maximizing, Locally Minimizing: Unsupervised Discriminant Projection with Applications to Face and Palm Biometrics , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[54]  Ben Kröse,et al.  The generative self-organizing map: a probabilistic generalization of Kohonen's SOM , 2002 .

[55]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[56]  Visar Berisha,et al.  Sparse Manifold Learning with Applications to SAR Image Classification , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.