Delineation of agricultural fields in smallholder farms from satellite images using fully convolutional networks and combinatorial grouping

Accurate spatial information of agricultural fields in smallholder farms is important for providing actionable information to farmers, managers, and policymakers. Very High Resolution (VHR) satellite images can capture such information. However, the automated delineation of fields in smallholder farms is a challenging task because of their small size, irregular shape and the use of mixed-cropping systems, which make their boundaries vaguely defined. Physical edges between smallholder fields are often indistinct in satellite imagery and contours need to be identified by considering the transition of the complex textural pattern between fields. In these circumstances, standard edge-detection algorithms fail to extract accurate boundaries. This article introduces a strategy to detect field boundaries using a fully convolutional network in combination with a globalisation and grouping algorithm. The convolutional network using an encoder-decoder structure is capable of learning complex spatial-contextual features from the image and accurately detects sparse field contours. A hierarchical segmentation is derived from the contours using the oriented watershed transform and by iteratively merging adjacent regions based on the average strength of their common boundary. Finally, field segments are obtained by adopting a combinatorial grouping algorithm exploiting the information of the segmentation hierarchy. An extensive experimental analysis is performed in two study areas in Nigeria and Mali using WorldView-2/3 images and comparing several state-of-the-art contour detection algorithms. The algorithms are compared based on the precision-recall accuracy assessment strategy which is tolerating small localisation errors in the detected contours. The proposed strategy shows promising results by automatically delineating field boundaries with F-scores higher than 0.7 and 0.6 on our two test areas, respectively, outperforming alternative techniques.

[1]  Jitendra Malik,et al.  Scale-invariant contour completion using conditional random fields , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[2]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Haipeng Wang,et al.  Target Classification Using the Deep Convolutional Networks for SAR Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Jie Liang,et al.  Spectral-spatial Feature Extraction for Hyperspectral Image Classification , 2016 .

[8]  Alfred Stein,et al.  Existential uncertainty of spatial objects segmented from satellite sensor imagery , 2002, IEEE Trans. Geosci. Remote. Sens..

[9]  Qing Liu,et al.  Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Alfred Stein,et al.  Detection of Informal Settlements from VHR Images Using Convolutional Neural Networks , 2017, Remote. Sens..

[11]  Xiao Xiang Zhu,et al.  A Self-Improving Convolution Neural Network for the Classification of Hyperspectral Data , 2016, IEEE Geoscience and Remote Sensing Letters.

[12]  Michael Ying Yang,et al.  Contour Detection for UAV-Based Cadastral Mapping , 2017, Remote. Sens..

[13]  Azriel Rosenfeld,et al.  Picture Processing and Psychopictorics , 1970 .

[14]  T. Raney,et al.  The Number, Size, and Distribution of Farms, Smallholder Farms, and Family Farms Worldwide , 2016 .

[15]  Lin Yan,et al.  Automated crop field extraction from multi-temporal Web Enabled Landsat Data , 2014 .

[16]  William J. Emery,et al.  Object-Based Convolutional Neural Network for High-Resolution Imagery Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[19]  Jordi Pont-Tuset,et al.  Supervised Evaluation of Image Segmentation and Object Proposal Techniques , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  O. Painter,et al.  Position-squared coupling in a tunable photonic crystal optomechanical cavity , 2015, 1505.07291.

[21]  Jitendra Malik,et al.  An empirical approach to grouping and segmentation , 2002 .

[22]  Alfred Stein,et al.  Recurrent Multiresolution Convolutional Networks for VHR Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Jordan Graesser,et al.  Detection of cropland field parcels from Landsat imagery , 2017 .

[24]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[25]  Zhuowen Tu,et al.  Supervised Learning of Edges and Object Boundaries , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Shihong Du,et al.  Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[28]  S. J. Oude Elberink,et al.  FULLY CONVOLUTIONAL NETWORKS FOR GROUND CLASSIFICATION FROM LIDAR POINT CLOUDS , 2018 .

[29]  Juan de Lara,et al.  Supporting user-oriented analysis for multi-view domain-specific visual languages , 2009, Inf. Softw. Technol..

[30]  Jordi Pont-Tuset,et al.  Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[32]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[33]  Nikhil Ketkar,et al.  Convolutional Neural Networks , 2021, Deep Learning with Python.

[34]  Un Desa Transforming our world : The 2030 Agenda for Sustainable Development , 2016 .

[35]  Gang Fu,et al.  Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network , 2017, Remote. Sens..

[36]  Yan Wang,et al.  DeepContour: A deep convolutional feature learned by positive-sharing loss for contour detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Alfred Stein,et al.  Deep Fully Convolutional Networks for the Detection of Informal Settlements in VHR Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[38]  Jianbo Shi,et al.  DeepEdge: A multi-scale bifurcated deep network for top-down contour detection , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Michele Volpi,et al.  Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[43]  Gunilla Borgefors,et al.  Integrated method for boundary delineation of agricultural fields in multispectral satellite images , 2000, IEEE Trans. Geosci. Remote. Sens..

[44]  C. Lawrence Zitnick,et al.  Fast Edge Detection Using Structured Forests , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Claudio Persello,et al.  A deep learning approach to the classification of sub-decimetre resolution aerial images , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[47]  Thomas J. Fuchs,et al.  A generalized computer vision approach to mapping crop fields in heterogeneous agricultural landscapes , 2016 .

[48]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Valentyn Tolpekin,et al.  A Workflow for Automated Satellite Image Processing: from Raw VHSR Data to Object-Based Spectral Information for Smallholder Agriculture , 2017, Remote. Sens..

[51]  Michele Volpi,et al.  Deep multi-task learning for a geographically-regularized semantic segmentation of aerial images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[52]  George Vosselman,et al.  A deep learning approach to DTM extraction from imagery using rule-based training labels , 2018 .

[53]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Jamie Sherrah,et al.  Semantic Labeling of Aerial and Satellite Imagery , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[55]  W. Beck,et al.  Hydrogen permeation in metals as a function of stress, temperature and dissolved hydrogen concentration , 1966, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[56]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[58]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Lei Guo,et al.  Exploring Hierarchical Convolutional Features for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Honglak Lee,et al.  Object Contour Detection with a Fully Convolutional Encoder-Decoder Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[62]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Zhuowen Tu,et al.  Holistically-Nested Edge Detection , 2015, ICCV.

[64]  Ramesh P. Singh Earth observation and sustainable development goals , 2020 .