Line-based image segmentation method: a new approach to segment VHSR remote sensing images automatically

ABSTRACT There exist different approaches for segmenting Very High Spatial Resolution (VHSR) remote sensing imagery with competitive performance, including object-based (e.g. Multiresolution), gradient-based (e.g. Watershed), and clustering-based (e.g. k-means) segmentation. However, they have a strong dependence on human assistance for tuning the required parameters (e.g. scale value, clusters number or tolerance thresholds), usually following a trial-and-error methodology that becomes tedious, hardly reproducible or transferable to other images, affecting negatively the methods’ robustness and efficiency. In this communication, we propose a novel method denominated Line-based segmentation (LBS) that automatically segments VHSR remote sensing imagery through a data-driven approach, bypassing the parameters’ definition by experts (i.e. region growing´s seeds and thresholds). The proposed algorithm offers flexibility and accuracy to segment regions with varying sizes and shapes, tested on different VHSR images, including multispectral images (WorldView-3, GeoEywe-1, Ikonos, QuickBird and SkySat), RGB aerial image (NAIP) and panchromatic image (Ikonos). The results revealed the LBS method shows a competitive performance compared against two well-known segmentation approaches, but without user intervention and generating consistent and repeatable segmentation results following an automatic fashion.

[1]  Dirk Tiede,et al.  ESP: a tool to estimate scale parameter for multiresolution image segmentation of remotely sensed data , 2010, Int. J. Geogr. Inf. Sci..

[2]  Stefan Lang,et al.  Object-based image analysis for remote sensing applications: modeling reality – dealing with complexity , 2008 .

[3]  Peter Hofmann Detecting buildings and roads from IKONOS data using additional elevation information , 2001 .

[4]  Emmanuel P. Baltsavias,et al.  Object extraction and revision by image analysis using existing geodata and knowledge: current status and steps towards operational systems☆ , 2004 .

[5]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[6]  Xiang Zhou,et al.  Evaluation of RGB, Color-Infrared and Multispectral Images Acquired from Unmanned Aerial Systems for the Estimation of Nitrogen Accumulation in Rice , 2018, Remote. Sens..

[7]  C. Burnett,et al.  A multi-scale segmentation/object relationship modelling methodology for landscape analysis , 2003 .

[8]  T. Kavzoglu,et al.  Parameter-Based Performance Analysis of Object-Based Image Analysis Using Aerial and Quikbird-2 Images , 2014 .

[9]  Garrison W. Cottrell,et al.  Color-to-Grayscale: Does the Method Matter in Image Recognition? , 2012, PloS one.

[10]  Philip Chan,et al.  Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[11]  Young-Gi Byun,et al.  A framework for the segmentation of high-resolution satellite imagery using modified seeded-region growing and region merging , 2011 .

[12]  R. Real,et al.  The Probabilistic Basis of Jaccard's Index of Similarity , 1996 .

[13]  Efkan Durmus,et al.  Optimizing the color-to-grayscale conversion for image classification , 2016, Signal Image Video Process..

[14]  Charles G. O'Hara,et al.  An object-based approach to detect road features for informal settlements near Sao Paulo, Brazil , 2008 .

[15]  David J. Fleet,et al.  Hamming Distance Metric Learning , 2012, NIPS.

[16]  Y. Zhang,et al.  A REVIEW ON IMAGE SEGMENTATION TECHNIQUES WITH REMOTE SENSING PERSPECTIVE , 2010 .

[17]  Thomas Blaschke,et al.  A comparison of three image-object methods for the multiscale analysis of landscape structure , 2003 .

[18]  Barbara Zitová,et al.  Performance evaluation of image segmentation algorithms on microscopic image data , 2015, Journal of microscopy.

[19]  P. Sivakumar,et al.  A REVIEW ON IMAGE SEGMENTATION TECHNIQUES , 2016 .

[20]  Tinku Acharya,et al.  Image processing , 2005 .

[21]  Arko Lucieer,et al.  Uncertainties in segmentation and their visualisation , 2004 .

[22]  Laurent Demagistri,et al.  Multispectral Satellite Image Processing , 2016 .

[23]  Águeda Arquero Hidalgo,et al.  Improving Parameters Selection of a Seeded Region Growing Method for Multiband Image Segmentation , 2015 .

[24]  A. P. Singh,et al.  Edge Detection in Gray Level Images based on the Shannon Entropy , 2008 .

[25]  C. Mallows,et al.  A Method for Comparing Two Hierarchical Clusterings , 1983 .

[26]  Mohamed Cheriet,et al.  CorrC2G: Color to Gray Conversion by Correlation , 2017, IEEE Signal Processing Letters.

[27]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[28]  Javier Sanchez Hernandez,et al.  Improving Parameters Selection of a Seeded Region Growing Method for Multiband Image Segmentation , 2015, IEEE Latin America Transactions.

[29]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Yu-Jin Zhang,et al.  Half Century for Image Segmentation , 2015 .

[31]  Huseyin Gokhan Akcay,et al.  Automatic Detection of Geospatial Objects Using Multiple Hierarchical Segmentations , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[32]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[33]  P. L. N. Raju,et al.  Road Network Extraction from High Resolution Multispectral Satellite Imagery Based on Object Oriented Techniques , 2014 .

[34]  Peijun Li,et al.  A new segmentation method for very high resolution imagery using spectral and morphological information , 2015 .

[35]  Adam Van Etten,et al.  You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery , 2018, ArXiv.

[36]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[37]  Narendra Ahuja,et al.  Multiscale image segmentation by integrated edge and region detection , 1997, IEEE Trans. Image Process..

[38]  Camille Roth,et al.  Natural Scales in Geographical Patterns , 2017, Scientific Reports.

[39]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[40]  Jianping Fan,et al.  Seeded region growing: an extensive and comparative study , 2005, Pattern Recognit. Lett..

[41]  W. Cao,et al.  The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XXXVIII-4/C7 CLASSIFICATION OF HIGH RESOLUTION OPTICAL AND SAR FUSION IMAGE USING FUZZY KNOWLEDGE AND OBJECT-ORIENTED PARADIGM , 2010 .

[42]  Christine Pohl,et al.  Multisensor image fusion in remote sensing: concepts, methods and applications , 1998 .

[43]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[44]  F. D. van der Meer,et al.  International institute for Geo - information Science and Earth Observation : ITC , 2009 .

[45]  Lei Guo,et al.  Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Arno Schäpe,et al.  Multiresolution Segmentation : an optimization approach for high quality multi-scale image segmentation , 2000 .

[47]  T. Blaschke TOWARDS A FRAMEWORK FOR CHANGE DETECTION BASED ON IMAGE OBJECTS , 2005 .

[48]  O. Csillik,et al.  Automated parameterisation for multi-scale image segmentation on multiple layers , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[49]  Frédéric Jurie,et al.  Vehicle detection in aerial imagery : A small target detection benchmark , 2016, J. Vis. Commun. Image Represent..

[50]  Shohreh Kasaei,et al.  Benign and malignant breast tumors classification based on region growing and CNN segmentation , 2015, Expert Syst. Appl..

[51]  Russell G. Congalton,et al.  A review of assessing the accuracy of classifications of remotely sensed data , 1991 .

[52]  U. Benz,et al.  Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information , 2004 .

[53]  Pierre Gançarski,et al.  Remote sensing image analysis by aggregation of segmentation-classification collaborative agents , 2018, Pattern Recognit..

[54]  Hui Zhang,et al.  Image segmentation evaluation: A survey of unsupervised methods , 2008, Comput. Vis. Image Underst..

[55]  Giorgos Mallinis,et al.  Assessment of Radiometric Resolution Impact on Remote Sensing Data Classification Accuracy , 2018, Remote. Sens..

[56]  Andrew Zisserman,et al.  Image Classification using Random Forests and Ferns , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[57]  Jaime S. Cardoso,et al.  Toward a generic evaluation of image segmentation , 2005, IEEE Transactions on Image Processing.

[58]  Pradeep Garg,et al.  The Effect of Radiometric Resolution on the Retrieval of Leaf Area Index from Agricultural Crops , 2006 .

[59]  Mariana Belgiu,et al.  Comparing supervised and unsupervised multiresolution segmentation approaches for extracting buildings from very high resolution imagery , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[60]  Frank Y. Shih,et al.  Retinal vessels segmentation based on level set and region growing , 2014, Pattern Recognit..

[61]  Mathias Schardt,et al.  Single tree detection in very high resolution remote sensing data , 2007 .

[62]  Gang Chen,et al.  Article in Press G Model International Journal of Applied Earth Observation and Geoinformation a Geobia Framework to Estimate Forest Parameters from Lidar Transects, Quickbird Imagery and Machine Learning: a Case Study in Quebec, Canada , 2022 .

[63]  Tinku Acharya,et al.  Image Processing: Principles and Applications , 2005, J. Electronic Imaging.

[64]  Martin Cadík,et al.  Perceptual Evaluation of Color‐to‐Grayscale Image Conversions , 2008, Comput. Graph. Forum.

[65]  Andrew P. Witkin,et al.  Scale-space filtering: A new approach to multi-scale description , 1984, ICASSP.

[66]  Li Feng,et al.  Adaptive Scale Selection for Multiscale Segmentation of Satellite Images , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[67]  Alessandro Montaghi,et al.  Accuracy assessment measures for image segmentation goodness of the Land Parcel Identification System (LPIS) in Denmark , 2013 .

[68]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[69]  P. Gong,et al.  Object-based Detailed Vegetation Classification with Airborne High Spatial Resolution Remote Sensing Imagery , 2006 .

[70]  Jorge Torres-Sánchez,et al.  An automatic object-based method for optimal thresholding in UAV images: Application for vegetation detection in herbaceous crops , 2015, Comput. Electron. Agric..

[71]  Qihao Weng,et al.  An Automated Method to Parameterize Segmentation Scale by Enhancing Intrasegment Homogeneity and Intersegment Heterogeneity , 2015, IEEE Geoscience and Remote Sensing Letters.

[72]  Jian Yang,et al.  A multi-band approach to unsupervised scale parameter selection for multi-scale image segmentation , 2014 .

[73]  X. Briottet,et al.  Shadow detection in very high spatial resolution aerial images: A comparative study , 2013 .

[74]  Patricia Gober,et al.  Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery , 2011, Remote Sensing of Environment.

[75]  Majida Albakoor,et al.  Region growing based segmentation algorithm for typewritten and handwritten text recognition , 2009, Appl. Soft Comput..

[76]  J. R. Jensen,et al.  An automatic region-based image segmentation algorithm for remote sensing applications , 2010, Environ. Model. Softw..

[77]  Junwei Han,et al.  A Survey on Object Detection in Optical Remote Sensing Images , 2016, ArXiv.

[78]  Li Feng,et al.  Toward Evaluating Multiscale Segmentations of High Spatial Resolution Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[79]  Travis Maxwell,et al.  A FUZZY LOGIC APPROACH TO SUPERVISED SEGMENTATION FOR OBJECT- ORIENTED CLASSIFICATION , 2006 .

[80]  Kai Zeng,et al.  Objective Quality Assessment for Color-to-Gray Image Conversion , 2015, IEEE Transactions on Image Processing.