Convex Formulation for Multiband Image Classification With Superpixel-Based Spatial Regularization

Superpixels are a powerful device to characterize the spatial–contextual information in remotely sensed hyperspectral image (HSI) interpretation. However, the exploitation of superpixels in classification problems is not straightforward, often leading to unbearable NP-hard discrete integer optimization problems. In this paper, we attack this hurdle by leveraging on a convex relaxation of the original integer optimization problem, which opens the door to include oversegmented superpixel-based regularizers. Specifically, we develop a new method for generating oversegmented superpixels. Then, we introduce a family of convex regularizers in the form of graph total variation, which promotes the same labeling in each superpixel. Vectorial total variation is also included in order to promote piecewise smoothness and align discontinuities along the class boundaries. The solution of the obtained convex optimization problem is computed with the split-augmented Lagrangian shrinkage algorithm. Experiments on HSIs yield classification maps with precise boundaries and inner consistency inside oversegmented superpixels, leading to the state-of-the-art classification accuracies.

[1]  Jelena Kovacevic,et al.  SegSALSA-STR: A convex formulation to supervised hyperspectral image segmentation using hidden fields and structure tensor regularization , 2015, 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[2]  Antonio J. Plaza,et al.  Semi-supervised hyperspectral image classification based on a Markov random field and sparse multinomial logistic regression , 2009, 2009 IEEE International Geoscience and Remote Sensing Symposium.

[3]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[4]  Daniel Cremers,et al.  A Survey and Comparison of Discrete and Continuous Multi-label Optimization Approaches for the Potts Model , 2013, International Journal of Computer Vision.

[5]  V. Jelen,et al.  Mobility tracking by interactive graph-cut segmentation with Bi-elliptical shape prior , 2010, 2010 IEEE 8th International Symposium on Applied Machine Intelligence and Informatics (SAMI).

[6]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Jon Atli Benediktsson,et al.  Multiple Spectral–Spatial Classification Approach for Hyperspectral Data , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Daphne Koller,et al.  Subproblem-Tree Calibration: A Unified Approach to Max-Product Message Passing , 2013, ICML.

[10]  Jon Atli Benediktsson,et al.  Spectral–Spatial Classification of Hyperspectral Imagery Based on Partitional Clustering Techniques , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Antonio J. Plaza,et al.  A Discontinuity Preserving Relaxation Scheme for Spectral–Spatial Hyperspectral Image Classification , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[13]  Ravi Bansal,et al.  Segmentation of Dynamic N-D Data Sets via Graph Cuts Using Markov Models , 2001, MICCAI.

[14]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Jon Atli Benediktsson,et al.  A spatial-spectral kernel-based approach for the classification of remote-sensing images , 2012, Pattern Recognit..

[16]  Antonio J. Plaza,et al.  Parallel and Distributed Dimensionality Reduction of Hyperspectral Data on Cloud Computing Architectures , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[18]  Shuai Yang,et al.  Superpixel-Based Classification Using K Distribution and Spatial Context for Polarimetric SAR Images , 2016, Remote. Sens..

[19]  Daniel Cremers,et al.  The Natural Vectorial Total Variation Which Arises from Geometric Measure Theory , 2012, SIAM J. Imaging Sci..

[20]  Petros Maragos,et al.  Convex Generalizations of Total Variation Based on the Structure Tensor with Applications to Inverse Problems , 2013, SSVM.

[21]  Chunming Li,et al.  Minimization of Region-Scalable Fitting Energy for Image Segmentation , 2008, IEEE Transactions on Image Processing.

[22]  Daniel Cremers,et al.  An Experimental Comparison of Discrete and Continuous Shape Optimization Methods , 2008, ECCV.

[23]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[24]  Pushmeet Kohli,et al.  Robust Higher Order Potentials for Enforcing Label Consistency , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[26]  Yu Jin Zhang,et al.  A review of recent evaluation methods for image segmentation , 2001, Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467).

[27]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[28]  L. Padma Suresh,et al.  Image segmentation using seeded region growing , 2012, 2012 International Conference on Computing, Electronics and Electrical Technologies (ICCEET).

[29]  Vijay K. Jain,et al.  Markov random field for tumor detection in digital mammography , 1995, IEEE Trans. Medical Imaging.

[30]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jon Atli Benediktsson,et al.  Spectral–Spatial Classification of Hyperspectral Images With a Superpixel-Based Discriminative Sparse Model , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Jelena Kovacevic,et al.  Alternating direction optimization for image segmentation using hidden Markov measure field models , 2014, Electronic Imaging.

[34]  William M. Wells,et al.  Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation , 2004, IEEE Transactions on Medical Imaging.

[35]  José M. F. Moura,et al.  Signal inpainting on graphs via total variation minimization , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Jelena Kovacevic,et al.  Supervised hyperspectral image segmentation: A convex formulation using hidden fields , 2014, 2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[37]  Edgar Arce Santana,et al.  Hidden Markov Measure Field Models for Image Segmentation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[39]  T. Chan,et al.  Fast dual minimization of the vectorial total variation norm and applications to color image processing , 2008 .

[40]  S. Thayammal,et al.  A Review On Segmentation Based I mage Compression Techniques , 2013 .

[41]  Daniel Cremers,et al.  A convex relaxation approach for computing minimal partitions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Filipe Condessa Robust Image Classification with Context and Rejection , 2016 .

[43]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Vladimir Kolmogorov,et al.  Minimizing Nonsubmodular Functions with Graph Cuts-A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Tien Yin Wong,et al.  Superpixel Classification Based Optic Disc and Optic Cup Segmentation for Glaucoma Screening , 2013, IEEE Transactions on Medical Imaging.

[46]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[47]  Jiankun Hu,et al.  Superpixel-Based Graphical Model for Remote Sensing Image Mapping , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Martin J. Wainwright,et al.  On the Optimality of Tree-reweighted Max-product Message-passing , 2005, UAI.

[50]  Antonio J. Plaza,et al.  Joint Sparse Sub-Pixel Mapping Model with Endmember Variability for Remotely Sensed Imagery , 2016, Remote. Sens..

[51]  Jan-Michael Frahm,et al.  Fast Global Labeling for Real-Time Stereo Using Multiple Plane Sweeps , 2008, VMV.

[52]  Jun Li,et al.  GPU Parallel Implementation of Spatially Adaptive Hyperspectral Image Classification , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[53]  Jon Atli Benediktsson,et al.  Segmentation and classification of hyperspectral images using watershed transformation , 2010, Pattern Recognit..

[54]  Antonio J. Plaza,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 1 Spectral–Spatial Classification of Hyperspectral Data Usi , 2022 .

[55]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[56]  Mário A. T. Figueiredo Bayesian image segmentation using wavelet-based priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[57]  Michele Volpi,et al.  Semantic segmentation of urban scenes by learning local class interactions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).