Deep Clustering With Intraclass Distance Constraint for Hyperspectral Images

The high dimensionality of hyperspectral images often results in the degradation of clustering performance. Due to the powerful ability of deep feature extraction and non-linear feature representation, the clustering algorithm based on deep learning has become a hot research topic in the field of hyperspectral remote sensing. However, most deep clustering algorithms for hyperspectral images utilize deep neural networks as feature extractor without considering prior knowledge constraints that are suitable for clustering. To solve this problem, we propose an intra-class distance constrained deep clustering algorithm for high-dimensional hyperspectral images. The proposed algorithm constrains the feature mapping procedure of the auto-encoder network by intra-class distance so that raw images are transformed from the original high-dimensional space to the low-dimensional feature space that is more conducive to clustering. Furthermore, the related learning process is treated as a joint optimization problem of deep feature extraction and clustering. Experimental results demonstrate the intense competitiveness of the proposed algorithm in comparison with state-of-the-art clustering methods of hyperspectral images.

[1]  Pierre Baldi,et al.  Autoencoders, Unsupervised Learning, and Deep Architectures , 2011, ICML Unsupervised and Transfer Learning.

[2]  Ce Zhang,et al.  Performance Evaluation of Cluster Validity Indices (CVIs) on Multi/Hyperspectral Remote Sensing Datasets , 2016, Remote. Sens..

[3]  Bo Du,et al.  Saliency-Guided Unsupervised Feature Learning for Scene Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Lei Zhu,et al.  Generating labeled samples for hyperspectral image classification using correlation of spectral bands , 2015, Frontiers of Computer Science.

[5]  Zahir Tari,et al.  A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis , 2014, IEEE Transactions on Emerging Topics in Computing.

[6]  Eugenio Culurciello,et al.  Convolutional Clustering for Unsupervised Learning , 2015, ArXiv.

[7]  Murray Shanahan,et al.  Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders , 2016, ArXiv.

[8]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[9]  Charles C. Kemp,et al.  A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder , 2017, IEEE Robotics and Automation Letters.

[10]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[11]  Xuan Tang,et al.  Reconstructible Nonlinear Dimensionality Reduction via Joint Dictionary Learning , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Qian Du,et al.  Unsupervised Hyperspectral Remote Sensing Image Clustering Based on Adaptive Density , 2018, IEEE Geoscience and Remote Sensing Letters.

[13]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Zhang Yi,et al.  Subspace clustering using a low-rank constrained autoencoder , 2018, Inf. Sci..

[15]  Yang Li-bin,et al.  Survey on Spectral Clustering Algorithms , 2008 .

[16]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[17]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[18]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[19]  Hongdong Li,et al.  Efficient dense subspace clustering , 2014, IEEE Winter Conference on Applications of Computer Vision.

[20]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[21]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Qiang Liu,et al.  A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture , 2018, IEEE Access.

[23]  Antonio J. Plaza,et al.  A New Sparse Subspace Clustering Algorithm for Hyperspectral Remote Sensing Imagery , 2017, IEEE Geoscience and Remote Sensing Letters.

[24]  Feiping Nie,et al.  Self-Weighted Supervised Discriminative Feature Selection , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[26]  Liangpei Zhang,et al.  Kernel Sparse Subspace Clustering with a Spatial Max Pooling Operation for Hyperspectral Remote Sensing Data Interpretation , 2017, Remote. Sens..

[27]  Xuelong Li,et al.  Embedded clustering via robust orthogonal least square discriminant analysis , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Lingfeng Wang,et al.  Deep Adaptive Image Clustering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[30]  Hongyuan Huo,et al.  An Enhanced IT2FCM* Algorithm Integrating Spectral Indices and Spatial Information for Multi-Spectral Remote Sensing Image Clustering , 2017, Remote. Sens..

[31]  Hao Shen,et al.  Trace Quotient with Sparsity Priors for Learning Low Dimensional Image Representations , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jon Atli Benediktsson,et al.  Multisource and Multitemporal Data Fusion in Remote Sensing , 2018, ArXiv.

[33]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Xin Huang,et al.  Unsupervised Deep Feature Learning for Urban Village Detection from High-Resolution Remote Sensing Images , 2017 .

[35]  Ohad Shamir,et al.  The Power of Depth for Feedforward Neural Networks , 2015, COLT.

[36]  Kai Wang,et al.  Sub-GAN: An Unsupervised Generative Model via Subspaces , 2018, ECCV.

[37]  Raquel Urtasun,et al.  Deep Spectral Clustering Learning , 2017, ICML.

[38]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Yinglong Dai,et al.  Analyzing Tongue Images Using a Conceptual Alignment Deep Autoencoder , 2018, IEEE Access.

[40]  René Vidal,et al.  Latent Space Sparse Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[41]  Francesca Bovolo,et al.  Unsupervised Multitemporal Spectral Unmixing for Detecting Multiple Changes in Hyperspectral Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Wei-Yun Yau,et al.  Deep Subspace Clustering with Sparsity Prior , 2016, IJCAI.

[43]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[44]  Naoto Yokoya,et al.  An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing , 2018, IEEE Transactions on Image Processing.

[45]  Tao Jiang,et al.  Enhanced IT2FCM algorithm using object-based triangular fuzzy set modeling for remote-sensing clustering , 2018, Comput. Geosci..

[46]  Liangpei Zhang,et al.  A Spatial Gaussian Mixture Model for Optical Remote Sensing Image Clustering , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[48]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[49]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[51]  Nicolas Gillis,et al.  Hierarchical Clustering of Hyperspectral Images Using Rank-Two Nonnegative Matrix Factorization , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Zhang Yi,et al.  Robust Subspace Clustering via Thresholding Ridge Regression , 2015, AAAI.

[53]  Carlos D. Castillo,et al.  Deep Density Clustering of Unconstrained Faces , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[54]  Junbin Gao,et al.  Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Jun Yu,et al.  Multitask Autoencoder Model for Recovering Human Poses , 2018, IEEE Transactions on Industrial Electronics.

[56]  Xian Wei,et al.  Learning Image and Video Representations Based on Sparsity Priors , 2017 .

[57]  René Vidal,et al.  Sparse Manifold Clustering and Embedding , 2011, NIPS.

[58]  Alessandro Laio,et al.  Clustering by fast search and find of density peaks , 2014, Science.

[59]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[60]  Nicolas Gillis,et al.  Fast and Robust Recursive Algorithmsfor Separable Nonnegative Matrix Factorization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Asif Ekbal,et al.  Multi-objective semi-supervised clustering for automatic pixel classification from remote sensing imagery , 2015, Soft Computing.

[62]  Karbhari V. Kale,et al.  A Research Review on Hyperspectral Data Processing and Analysis Algorithms , 2017, Proceedings of the National Academy of Sciences, India Section A: Physical Sciences.

[63]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[64]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[65]  Murat Kantarcioglu,et al.  Adversarial Clustering: A Grid Based Clustering Algorithm Against Active Adversaries , 2018, ArXiv.

[66]  Dorde T. Grozdic,et al.  Whispered Speech Recognition Using Deep Denoising Autoencoder and Inverse Filtering , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[67]  Mauro Maggioni,et al.  Unsupervised Clustering and Active Learning of Hyperspectral Images With Nonlinear Diffusion , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[68]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[69]  Mauro Maggioni,et al.  Learning by Unsupervised Nonlinear Diffusion , 2018, J. Mach. Learn. Res..

[70]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[71]  Chuang Sun,et al.  Deep Coupling Autoencoder for Fault Diagnosis With Multimodal Sensory Data , 2018, IEEE Transactions on Industrial Informatics.

[72]  V. D. Sa.  Spectral Clustering with Two Views , 2007 .

[73]  Jiashi Feng,et al.  Deep Adversarial Subspace Clustering , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[74]  Lorenzo Bruzzone,et al.  Learning Discriminative Embedding for Hyperspectral Image Clustering Based on Set-to-Set and Sample-to-Sample Distances , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[75]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  S. Margret Anouncia,et al.  Unsupervised Segmentation of Remote Sensing Images using FD Based Texture Analysis Model and ISODATA , 2017, Int. J. Ambient Comput. Intell..

[77]  Zhuo Chen,et al.  Deep clustering: Discriminative embeddings for segmentation and separation , 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[78]  Meirav Galun,et al.  Fundamental Limitations of Spectral Clustering , 2006, NIPS.

[79]  Kun Hou,et al.  Two-Stage Clustering Technique Based on the Neighboring Union Histogram for Hyperspectral Remote Sensing Images , 2017, IEEE Access.

[80]  Ohad Shamir,et al.  Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks , 2016, ICML.

[81]  Tengyu Ma,et al.  On the Ability of Neural Nets to Express Distributions , 2017, COLT.

[82]  Linear Autoencoder Networks for Structured Data , 2013 .

[83]  Qi Wang,et al.  Fast Spectral Clustering for Unsupervised Hyperspectral Image Classification , 2019, Remote. Sens..

[84]  Junbin Gao,et al.  Robust latent low rank representation for subspace clustering , 2014, Neurocomputing.

[85]  Liangpei Zhang,et al.  Spectral–Spatial Sparse Subspace Clustering for Hyperspectral Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[86]  Feng Liu,et al.  Auto-encoder Based Data Clustering , 2013, CIARP.

[87]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[88]  Hao Shen,et al.  Trace Quotient Meets Sparsity: A Method for Learning Low Dimensional Image Representations , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).