Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis

The success of deep learning in visual recognition tasks has driven advancements in multiple fields of research. Particularly, increasing attention has been drawn towards its application in agriculture. Nevertheless, while visual pattern recognition on farmlands carries enormous economic values, little progress has been made to merge computer vision and crop sciences due to the lack of suitable agricultural image datasets. Meanwhile, problems in agriculture also pose new challenges in computer vision. For example, semantic segmentation of aerial farmland images requires inference over extremely large-size images with extreme annotation sparsity. These challenges are not present in most of the common object datasets, and we show that they are more challenging than many other aerial image datasets. To encourage research in computer vision for agriculture, we present Agriculture-Vision: a large-scale aerial farmland image dataset for semantic segmentation of agricultural patterns. We collected 94,986 high-quality aerial images from 3,432 farmlands across the US, where each image consists of RGB and Near-infrared (NIR) channels with resolution as high as 10 cm per pixel. We annotate nine types of field anomaly patterns that are most important to farmers. As a pilot study of aerial agricultural semantic segmentation, we perform comprehensive experiments using popular semantic segmentation models; we also propose an effective model designed for aerial agricultural pattern recognition. Our experiments demonstrate several challenges Agriculture-Vision poses to both the computer vision and agriculture communities. Future versions of this dataset will include even more aerial images, anomaly patterns and image channels.

[1]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  John A. Marchant,et al.  Evaluation of an imaging sensor for detecting vegetation using different waveband combinations , 2001 .

[3]  Bolei Zhou,et al.  Scene Parsing through ADE20K Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  D. K. Giles,et al.  Precision weed control system for cotton , 2002 .

[5]  Xiao Chen,et al.  FOAL: Fast Online Adaptive Learning for Cardiac Motion Estimation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Arko Lucieer,et al.  Sensor Correction of a 6-Band Multispectral Imaging Sensor for UAV Remote Sensing , 2012, Remote. Sens..

[7]  Agnès Bégué,et al.  Can Commercial Digital Cameras Be Used as Multispectral Sensors? A Crop Monitoring Test , 2008, Sensors.

[8]  Jinjun Xiong,et al.  Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Xin Zhao,et al.  Use of Unmanned Aerial Vehicle Imagery and Deep Learning UNet to Extract Rice Lodging , 2019, Sensors.

[10]  Yunchao Wei,et al.  Weakly Supervised Scene Parsing with Point-based Distance Metric Learning , 2018, AAAI.

[11]  Shashi Shekhar,et al.  NDVI Versus CNN Features in Deep Learning for Land Cover Clasification of Aerial Images , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[12]  Cyrill Stachniss,et al.  REAL-TIME BLOB-WISE SUGAR BEETS VS WEEDS CLASSIFICATION FOR MONITORING FIELDS USING CONVOLUTIONAL NEURAL NETWORKS , 2017 .

[13]  Thenkurussi Kesavadas,et al.  A Novel Framework for 3D-2D Vertebra Matching , 2019, 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR).

[14]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  V. Sowmya,et al.  Deep learning architectures for land cover classification using red and near-infrared satellite images , 2019, Multimedia Tools and Applications.

[16]  Kilian Q. Weinberger,et al.  Convolutional Networks with Dense Connectivity , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Jiri Matas,et al.  Systematic evaluation of convolution neural network advances on the Imagenet , 2017, Comput. Vis. Image Underst..

[18]  Jinjun Xiong,et al.  Decoupled Classification Refinement: Hard False Positive Suppression for Object Detection , 2018, ArXiv.

[19]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[20]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[21]  G. Meyer,et al.  Color indices for weed identification under various soil, residue, and lighting conditions , 1994 .

[22]  Chao Liu,et al.  Cultivated land information extraction in UAV imagery based on deep convolutional neural network and transfer learning , 2017, Journal of Mountain Science.

[23]  A. Gitelson,et al.  Novel algorithms for remote estimation of vegetation fraction , 2002 .

[24]  N. Haala,et al.  PERFORMANCE TEST ON UAV-BASED PHOTOGRAMMETRIC DATA COLLECTION , 2012 .

[25]  Dallas E. Peterson,et al.  WEED DETECTION USING COLOR MACHINE VISION , 2000 .

[26]  Jinjun Xiong,et al.  SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems , 2020, MLSys.

[27]  Jörn Ostermann,et al.  A Crop/Weed Field Image Dataset for the Evaluation of Computer Vision Based Precision Agriculture Tasks , 2014, ECCV Workshops.

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Rogério Schmidt Feris,et al.  SpotTune: Transfer Learning Through Adaptive Fine-Tuning , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Honghui Shi,et al.  Geometry-Aware Traffic Flow Analysis by Detection and Tracking , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[33]  Bin Xiao,et al.  Bottom-up Higher-Resolution Networks for Multi-Person Pose Estimation , 2019, ArXiv.

[34]  Jing Huang,et al.  DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[35]  Rynson W. H. Lau,et al.  Geometry-Aware Distillation for Indoor Semantic Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  P. Zarco-Tejada,et al.  A PRI-based water stress index combining structural and chlorophyll effects: Assessment using diurnal narrow-band airborne imagery and the CWSI thermal index , 2013 .

[38]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[40]  Ling Shao,et al.  iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images , 2019, CVPR Workshops.

[41]  C. Langlotz,et al.  Performance of a Deep-Learning Neural Network Model in Assessing Skeletal Maturity on Pediatric Hand Radiographs. , 2017, Radiology.

[42]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[43]  A. Rango,et al.  Image Processing and Classification Procedures for Analysis of Sub-decimeter Imagery Acquired with an Unmanned Aircraft over Arid Rangelands , 2011 .

[44]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Jinjun Xiong,et al.  SPGNet: Semantic Prediction Guidance for Scene Parsing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  Yunchao Wei,et al.  Self-Similarity Grouping: A Simple Unsupervised Cross Domain Adaptation Approach for Person Re-Identification , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[47]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[48]  Héctor F. Satizábal,et al.  Augmenting a convolutional neural network with local histograms - A case study in crop classification from high-resolution UAV imagery , 2016, ESANN.

[49]  Kaiming He,et al.  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour , 2017, ArXiv.

[50]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Andreas Dengel,et al.  EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[52]  Mostafa Rahimi Azghadi,et al.  DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning , 2018, Scientific Reports.

[53]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[54]  Thomas S. Huang,et al.  Any-Precision Deep Neural Networks , 2019, ArXiv.

[55]  Yunchao Wei,et al.  CCNet: Criss-Cross Attention for Semantic Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[56]  A. Aniyan,et al.  Classifying Radio Galaxies with the Convolutional Neural Network , 2017, 1705.03413.

[57]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[58]  Supratik Mukhopadhyay,et al.  DeepSat: a learning framework for satellite imagery , 2015, SIGSPATIAL/GIS.

[59]  Craig S. T. Daughtry,et al.  Acquisition of NIR-Green-Blue Digital Photographs from Unmanned Aircraft for Crop Monitoring , 2010, Remote. Sens..

[60]  Thomas S. Huang,et al.  Computed tomography super-resolution using convolutional neural networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[61]  Jinjun Xiong,et al.  Revisiting Pre-training: An Efficient Training Method for Image Classification. , 2018 .

[62]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[63]  Pierre Alliez,et al.  Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).