A Hybrid Markov Random Field Model With Multi-Granularity Information for Semantic Segmentation of Remote Sensing Imagery

High-spatial-resolution (HSR) remote sensing images usually contain rich hierarchical semantic information. However, many methods fail to solve the segmentation misclassification problems for HSR images due to just considering one layer of granularity information, such as the pixel granularity layer or the object granularity layer. This paper presents a hybrid Markov random field model for the semantic segmentation of HSR images by paying closer attention to capture the multi-granularity information. In this model, a probability graph with a multilayer structure is first built to represent different granularities information. Then, a hybrid label field is developed to model the multi-granularity classes with the form of a vector, and a new joint distribution is designed to capture the isotropic spatial interactions within each layer of the hybrid label field and the anisotropic spatial interactions between different layers. A generative probabilistic inference is proposed to realize the synergy between the multi-granularity information and the hybrid label interactions by iteratively updating the likelihood function and the joint probability of the hybrid label. The final semantic segmentation result can be achieved when the probabilistic inference converges. Experimental results over different HSR remote sensing images show that the proposed method can achieve more accurate segmentation than other state-of-the-art methods.

[1]  Yun Zhang,et al.  Semantic Segmentation of Remote Sensing Imagery Using an Object-Based Markov Random Field Model With Auxiliary Label Fields , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Xiaofeng Wang,et al.  A new localized superpixel Markov random field for image segmentation , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[3]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Leiguang Wang,et al.  A Markov random field integrating spectral dissimilarity and class co-occurrence dependency for remote sensing image classification optimization , 2017 .

[5]  Zhe Zhao,et al.  Image Fuzzy Clustering Based on the Region-Level Markov Random Field Model , 2015, IEEE Geoscience and Remote Sensing Letters.

[6]  Shuzhi Sam Ge,et al.  Image tag completion via dual-view linear sparse reconstructions , 2014, Comput. Vis. Image Underst..

[7]  Francesca Bovolo,et al.  A Novel Technique Based on Deep Learning and a Synthetic Target Database for Classification of Urban Areas in PolSAR Data , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Xiaoqiang Lu,et al.  Scene Recognition by Manifold Regularized Deep Learning Architecture , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Michal Haindl,et al.  Texture segmentation benchmark , 2008, 2008 19th International Conference on Pattern Recognition.

[10]  Lei Guo,et al.  Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Chen Zheng,et al.  Image segmentation using a unified Markov random field model , 2017, IET Image Process..

[12]  Martial Hebert,et al.  Measures of Similarity , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[13]  Brian P. Salmon,et al.  A Markov Random Field model for decision level fusion of multi-source image segments , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[14]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[15]  Haluk Derin,et al.  Modeling and Segmentation of Noisy and Textured Images Using Gibbs Random Fields , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Daoqiang Zhang,et al.  Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Zhi-Qiang Liu,et al.  Self-Validated Labeling of Markov Random Fields for Image Segmentation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Yansheng Li,et al.  The edge-preservation multi-classifier relearning framework for the classification of high-resolution remotely sensed imagery , 2018 .

[20]  David A. Clausi,et al.  IRGS: Image Segmentation Using Edge Penalties and Region Growing , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[22]  Qiang Ji,et al.  Image Segmentation with a Unified Graphical Model , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[24]  Ling Shao,et al.  Sub-Markov Random Walk for Image Segmentation , 2016, IEEE Transactions on Image Processing.

[25]  Liangpei Zhang,et al.  An SVM Ensemble Approach Combining Spectral, Structural, and Semantic Features for the Classification of High-Resolution Remotely Sensed Imagery , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[27]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Geoffrey J. Hay,et al.  Geographic Object-Based Image Analysis (GEOBIA): A new name for a new discipline , 2008 .

[29]  Liangpei Zhang,et al.  An Adaptive Mean-Shift Analysis Approach for Object Extraction and Classification From Urban Hyperspectral Imagery , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Bo Du,et al.  Spatial Coherence-Based Batch-Mode Active Learning for Remote Sensing Image Classification , 2015, IEEE Transactions on Image Processing.

[31]  Jake Porway,et al.  A hierarchical and contextual model for aerial image understanding , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Hong Sun,et al.  An Unsupervised Segmentation Method Using Markov Random Field on Region Adjacency Graph for SAR Images , 2006, 2006 CIE International Conference on Radar.

[33]  Amy Loutfi,et al.  Classification and Segmentation of Satellite Orthoimagery Using Convolutional Neural Networks , 2016, Remote. Sens..

[34]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[36]  David A. Clausi,et al.  SAR Sea-Ice Image Analysis Based on Iterative Region Growing Using Semantics , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Xueming Qian,et al.  Semantic Annotation of High-Resolution Satellite Images via Weakly Supervised Learning , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[39]  Hideki Noda,et al.  MRF-based texture segmentation using wavelet decomposed images , 2000, Electronic Imaging.

[40]  Jun Li,et al.  A Novel MRF-Based Multifeature Fusion for Classification of Remote Sensing Images , 2016, IEEE Geoscience and Remote Sensing Letters.

[41]  David A. Clausi,et al.  Unsupervised Polarimetric SAR Image Segmentation and Classification Using Region Growing With Edge Penalty , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Jake Porway,et al.  A Hierarchical and Contextual Model for Aerial Image Parsing , 2010, International Journal of Computer Vision.

[43]  Lin He,et al.  Superpixel-Based Semisupervised Active Learning for Hyperspectral Image Classification , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[44]  Lei Guo,et al.  Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Bin Zhang,et al.  Region-based classification by combining MS segmentation and MRF for POLSAR images , 2013 .

[46]  William J. Emery,et al.  Object-Based Convolutional Neural Network for High-Resolution Imagery Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Fang Liu,et al.  Unsupervised Deep Feature Learning for Remote Sensing Image Retrieval , 2018, Remote. Sens..

[49]  Qi Tian,et al.  Image Annotation by Latent Community Detection and Multikernel Learning , 2015, IEEE Transactions on Image Processing.

[50]  Chen Zheng,et al.  Semantic Segmentation of Remote Sensing Imagery Using Object-Based Markov Random Field Model With Regional Penalties , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[51]  Liang-pei Zhang,et al.  Spectral-spatial classification of hyperspectral imagery with cooperative game , 2018 .

[52]  Rachid Deriche,et al.  Geodesic Active Contours and Level Sets for the Detection and Tracking of Moving Objects , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Ryuei Nishii.  A Markov random field-based approach to decision-level fusion for remote sensing image classification , 2003, IEEE Trans. Geosci. Remote. Sens..