Computer Vision with Deep Learning for Plant Phenotyping in Agriculture: A Survey

In light of growing challenges in agriculture with ever growing food demand across the world, efficient crop management techniques are necessary to increase crop yield. Precision agriculture techniques allow the stakeholders to make effective and customized crop management decisions based on data gathered from monitoring crop environments. Plant phenotyping techniques play a major role in accurate crop monitoring. Advancements in deep learning have made previously difficult phenotyping tasks possible. This survey aims to introduce the reader to the state of the art research in deep plant phenotyping.

[1]  Tristan Perez,et al.  DeepFruits: A Fruit Detection System Using Deep Neural Networks , 2016, Sensors.

[2]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[3]  Genggeng Liu,et al.  A Survey of Object Co-Segmentation , 2019, IEEE Access.

[4]  Henry Medeiros,et al.  Detecting Invasive Insects with Unmanned Aerial Vehicles , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[5]  Wei Guo,et al.  Aerial Imagery Analysis – Quantifying Appearance and Number of Sorghum Heads for Applications in Breeding and Agronomy , 2018, Front. Plant Sci..

[6]  Hanno Scharr,et al.  ARIGAN: Synthetic Arabidopsis Plants using Generative Adversarial Network , 2017, bioRxiv.

[7]  Rafael Rieder,et al.  Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review , 2018, Comput. Electron. Agric..

[8]  Loreto Susperregi,et al.  A Benchmarking of Learning Strategies for Pest Detection and Identification on Tomato Plants for Autonomous Scouting Robots Using Internal Databases , 2019, J. Sensors.

[9]  Hanno Scharr,et al.  Annotated Image Datasets of Rosette Plants , 2014 .

[10]  Jean Ponce,et al.  A Theoretical Analysis of Feature Pooling in Visual Recognition , 2010, ICML.

[11]  Medhat A. Moussa,et al.  The Ciona17 Dataset for Semantic Segmentation of Invasive Species in a Marine Aquaculture Environment , 2017, 2017 14th Conference on Computer and Robot Vision (CRV).

[12]  Masahiro Miyazaki,et al.  Study on Electrostatic Pesticide Spraying System for Low-Concentration, High-Volume Applications , 2017 .

[13]  Roland Siegwart,et al.  weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming , 2017, IEEE Robotics and Automation Letters.

[14]  Dan Wu,et al.  Panicle-SEG: a robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization , 2017, Plant Methods.

[15]  Diane Rowland,et al.  Overcoming Small Minirhizotron Datasets Using Transfer Learning , 2020, Comput. Electron. Agric..

[16]  Shreelekha Pandey,et al.  Semi-automatic leaf disease detection and classification system for soybean culture , 2018, IET Image Process..

[17]  Dong Sun Park,et al.  High-Performance Deep Neural Network-Based Tomato Plant Diseases and Pests Diagnosis System With Refinement Filter Bank , 2018, Front. Plant Sci..

[18]  Yubin Lan,et al.  Development and prospect of unmanned aerial vehicle technologies for agricultural production management , 2013 .

[19]  Konstantinos P. Ferentinos,et al.  Deep learning models for plant disease detection and diagnosis , 2018, Comput. Electron. Agric..

[20]  Daniel K. Nkemelu,et al.  Deep Convolutional Neural Network for Plant Seedlings Classification , 2018, ArXiv.

[21]  Luciano Vieira Koenigkan,et al.  A Study on the Detection of Cattle in UAV Images Using Deep Learning , 2019, Sensors.

[22]  Ian Goodfellow,et al.  Generative adversarial networks , 2020, Commun. ACM.

[23]  Joey N. Shaw,et al.  Evaluation of multispectral data for rapid assessment of wheat straw residue cover , 2004 .

[24]  Marcel Salathé,et al.  Using Deep Learning for Image-Based Plant Disease Detection , 2016, Front. Plant Sci..

[25]  Yunchao Wei,et al.  Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  K. R. Aravind,et al.  Grape Crop Disease Classification Using Transfer Learning Approach , 2018, Proceedings of the International Conference on ISMAC in Computational Vision and Bio-Engineering 2018 (ISMAC-CVB).

[27]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[28]  Yao Zhou,et al.  A Vision-Based Counting and Recognition System for Flying Insects in Intelligent Agriculture , 2018, Sensors.

[29]  Zoubin Ghahramani,et al.  Deep Bayesian Active Learning with Image Data , 2017, ICML.

[30]  Pedro A Sanchez,et al.  Cutting World Hunger in Half , 2005, Science.

[31]  P. R. Scott,et al.  Plant disease: a threat to global food security. , 2005, Annual review of phytopathology.

[32]  Wei Guo,et al.  An Adaptive Supervision Framework for Active Learning in Object Detection , 2019, BMVC.

[33]  Frédéric Baret,et al.  Ear density estimation from high resolution RGB imagery using deep learning technique , 2019, Agricultural and Forest Meteorology.

[34]  J. Flexas,et al.  UAVs challenge to assess water stress for sustainable agriculture , 2015 .

[35]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[36]  P. Supriya,et al.  Computer vision based segregation of carrot and curry leaf plants with weed identification in carrot field , 2017, 2017 International Conference on Computing Methodologies and Communication (ICCMC).

[37]  Naman Jain,et al.  PlantDoc: A Dataset for Visual Plant Disease Detection , 2019, COMAD/CODS.

[38]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[39]  Guoqian Chen,et al.  An overview of arable land use for the world economy: From source to sink via the global supply chain , 2018, Land Use Policy.

[40]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[41]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[42]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[43]  Pejman Rasti,et al.  Data Augmentation From RGB to Chlorophyll Fluorescence Imaging Application to Leaf Segmentation of Arabidopsis Thaliana From Top View Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[44]  Ron Kimmel,et al.  Data Augmentation for Leaf Segmentation and Counting Tasks in Rosette Plants , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[45]  Paolo Valigi,et al.  Combining Domain Adaptation and Spatial Consistency for Unseen Fruits Counting: A Quasi-Unsupervised Approach , 2020, IEEE Robotics and Automation Letters.

[46]  Using satellite imagery to asses trends in soil and crop productivity across landscapes , 2015 .

[47]  Zhu-Hong You,et al.  Leaf image based cucumber disease recognition using sparse representation classification , 2017, Comput. Electron. Agric..

[48]  E. Fereres,et al.  Using high resolution UAV thermal imagery to assess the variability in the water status of five fruit tree species within a commercial orchard , 2013, Precision Agriculture.

[49]  Pablo J. Zarco-Tejada,et al.  High-resolution airborne hyperspectral and thermal imagery for early detection of Verticillium wilt of olive using fluorescence, temperature and narrow-band spectral indices , 2013 .

[50]  Mitchell C. Hunter,et al.  Agriculture in 2050: Recalibrating Targets for Sustainable Intensification , 2017 .

[51]  Yong Bai,et al.  Recognition of Pyralidae Insects Using Intelligent Monitoring Autonomous Robot Vehicle in Natural Farm Scene , 2019, ArXiv.

[52]  Laura A. Zanella-Calzada,et al.  Comparison of Convolutional Neural Network Architectures for Classification of Tomato Plant Diseases , 2020, Applied Sciences.

[53]  Przemyslaw Prusinkiewicz,et al.  The use of plant models in deep learning: an application to leaf counting in rosette plants , 2018, Plant Methods.

[54]  Joaquin Vanschoren,et al.  Data Augmentation using Conditional Generative Adversarial Networks for Leaf Counting in Arabidopsis Plants , 2018, BMVC.

[55]  Min Zuo,et al.  CropDeep: The Crop Vision Dataset for Deep-Learning-Based Classification and Detection in Precision Agriculture , 2019, Sensors.

[56]  K. Siddique,et al.  Regulated deficit irrigation for crop production under drought stress. A review , 2016, Agronomy for Sustainable Development.

[57]  Jin Chen,et al.  Multi-modality imagery database for plant phenotyping , 2016, Machine Vision and Applications.

[58]  Zhiguo Cao,et al.  Joint crop and tassel segmentation in the wild , 2015, 2015 Chinese Automation Congress (CAC).

[59]  Alejandro Vergara,et al.  AI-powered banana diseases and pest detection , 2019, Plant Methods.

[60]  Hanno Scharr,et al.  Finely-grained annotated datasets for image-based plant phenotyping , 2016, Pattern Recognit. Lett..

[61]  Hamid Laga,et al.  Detection and analysis of wheat spikes using Convolutional Neural Networks , 2018, Plant Methods.

[62]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[63]  Seishi Ninomiya,et al.  Automatic estimation of heading date of paddy rice using deep learning , 2019, Plant Methods.

[64]  C. Heald,et al.  Threat to future global food security from climate change and ozone air pollution , 2014 .

[65]  Austin M. Jensen,et al.  Assessment of Surface Soil Moisture Using High-Resolution Multi-Spectral Imagery and Artificial Neural Networks , 2015, Remote. Sens..

[66]  Ji Liu,et al.  Data Enhancement for Plant Disease Classification Using Generated Lesions , 2020 .

[67]  Darko Stefanovic,et al.  Deep Neural Networks Based Recognition of Plant Diseases by Leaf Image Classification , 2016, Comput. Intell. Neurosci..

[68]  Ramjee Prasad,et al.  Data collection using miniature aerial vehicles in wireless sensor networks , 2016, IET Wirel. Sens. Syst..

[69]  Pierre Beauseroy,et al.  Weakly-supervised learning approach for potato defects segmentation , 2019, Eng. Appl. Artif. Intell..

[70]  Gonzalo Pajares,et al.  Support Vector Machines for crop/weeds identification in maize fields , 2012, Expert Syst. Appl..

[71]  Andrea Luvisi,et al.  X-FIDO: An Effective Application for Detecting Olive Quick Decline Syndrome with Deep Learning and Data Fusion , 2017, Front. Plant Sci..

[72]  A. B. Harto,et al.  Generating soil salinity, soil moisture, soil pH from satellite imagery and its analysis , 2020, Information Processing in Agriculture.

[73]  Alessandro Matese,et al.  A flexible unmanned aerial vehicle for precision agriculture , 2012, Precision Agriculture.

[74]  Wei Wu,et al.  Detection of aphids in wheat fields using a computer vision technique , 2016 .

[75]  G. Palm.  Warren McCulloch and Walter Pitts: A Logical Calculus of the Ideas Immanent in Nervous Activity , 1986 .

[76]  Aibin Chen,et al.  Rapid Detection of Rice Disease Based on FCM-KM and Faster R-CNN Fusion , 2019, IEEE Access.

[77]  Joachim Denzler,et al.  Active Learning for Deep Object Detection , 2018, VISIGRAPP.

[78]  Hitoshi Iyatomi,et al.  LeafGAN: An Effective Data Augmentation Method for Practical Plant Disease Diagnosis , 2020, IEEE Transactions on Automation Science and Engineering.

[79]  Marcel Salathé,et al.  An open access repository of images on plant health to enable the development of mobile disease diagnostics through machine learning and crowdsourcing , 2015, ArXiv.

[80]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[81]  Application of Satellite Remote Sensing to find Soil Fertilization by using Soil Colour , 2013 .

[82]  Shuyuan Yang,et al.  A Survey of Deep Learning-Based Object Detection , 2019, IEEE Access.

[83]  Jörn Ostermann,et al.  A Crop/Weed Field Image Dataset for the Evaluation of Computer Vision Based Precision Agriculture Tasks , 2014, ECCV Workshops.

[84]  Gui-Song Xia,et al.  Mini-UAV-based Remote Sensing: Techniques, Applications and Prospectives , 2018, ArXiv.

[85]  Sachin Malpe,et al.  Automated leaf disease detection and treatment recommendation using Transfer Learning , 2019 .

[86]  Wolfram Burgard,et al.  Building an Aerial–Ground Robotics System for Precision Farming: An Adaptable Solution , 2019, IEEE Robotics & Automation Magazine.

[87]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[88]  Tianzhu Xiang,et al.  Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, applications, and prospects , 2019, IEEE Geoscience and Remote Sensing Magazine.

[89]  Dong Sun Park,et al.  Unsupervised Image Translation using Adversarial Networks for Improved Plant Disease Recognition , 2019, Comput. Electron. Agric..

[90]  Mahsa Asnafi,et al.  A Review on Potential Applications of Unmanned Aerial Vehicle for Construction Industry , 2018 .

[91]  Benoit Aubert,et al.  IT as enabler of sustainable farming: An empirical analysis of farmers' adoption decision of precision agriculture technology , 2012, Decis. Support Syst..

[92]  Chris Yakopcic,et al.  A State-of-the-Art Survey on Deep Learning Theory and Architectures , 2019, Electronics.

[93]  Ruimao Zhang,et al.  Cost-Effective Active Learning for Deep Image Classification , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[94]  Johanna Link,et al.  Combined Spectral and Spatial Modeling of Corn Yield Based on Aerial Images and Crop Surface Models Acquired with an Unmanned Aircraft System , 2014, Remote. Sens..

[95]  Mercelin Francis,et al.  Disease Detection and Classification in Agricultural Plants Using Convolutional Neural Networks — A Visual Understanding , 2019, 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN).

[96]  P. Singh,et al.  Assessment of yield and economic losses in agriculture due to weeds in India , 2018 .

[97]  Radhika Dave,et al.  Extreme vulnerability of smallholder farmers to agricultural risks and climate change in Madagascar , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[98]  Won Suk Lee,et al.  Comparison of two aerial imaging platforms for identification of Huanglongbing-infected citrus trees , 2013 .

[99]  Silvio Savarese,et al.  Active Learning for Convolutional Neural Networks: A Core-Set Approach , 2017, ICLR.

[100]  Muhammad Hafeez Javed,et al.  K-means Based Automatic Pests Detection and Classification for Pesticides Spraying , 2017 .

[101]  Troy Jensen,et al.  Crop maturity mapping using a low-cost low-altitude remote sensing system , 2009 .

[102]  Paolo Valigi,et al.  Weakly Supervised Fruit Counting for Yield Estimation Using Spatial Consistency , 2019, IEEE Robotics and Automation Letters.

[103]  Hashmat Shadab Malik,et al.  Disease Recognition in Sugarcane Crop Using Deep Learning , 2020 .

[104]  Mostafa Mehdipour-Ghazi,et al.  Plant identification using deep neural networks via optimization of transfer learning parameters , 2017, Neurocomputing.

[105]  Morten Bisgaard,et al.  Adaptive Surveying and Early Treatment of Crops with a Team of Autonomous Vehicles , 2011, ECMR.

[106]  Kristen Grauman,et al.  Large-Scale Live Active Learning: Training Object Detectors with Crawled Data and Crowds , 2011, CVPR 2011.

[107]  Baskar Ganapathysubramanian,et al.  A Weakly Supervised Deep Learning Framework for Sorghum Head Detection and Counting , 2019, Plant phenomics.

[108]  Saraju P. Mohanty,et al.  dCrop: A Deep-Learning Based Framework for Accurate Prediction of Diseases of Crops in Smart Agriculture , 2019, 2019 IEEE International Symposium on Smart Electronic Systems (iSES) (Formerly iNiS).

[109]  José G. M. Esgario,et al.  Deep Learning for Classification and Severity Estimation of Coffee Leaf Biotic Stress , 2019, Comput. Electron. Agric..

[110]  Fangyuan Wang,et al.  A Recognition Method for Rice Plant Diseases and Pests Video Detection Based on Deep Convolutional Neural Network , 2020, Sensors.

[111]  Daniele Nardi,et al.  Crop and Weeds Classification for Precision Agriculture Using Context-Independent Pixel-Wise Segmentation , 2019, 2019 Third IEEE International Conference on Robotic Computing (IRC).

[112]  Ana Allende,et al.  Irrigation Water Quality for Leafy Crops: A Perspective of Risks and Potential Solutions , 2015, International journal of environmental research and public health.

[113]  Darko Stefanovic,et al.  Solving Current Limitations of Deep Learning Based Approaches for Plant Disease Detection , 2019, Symmetry.

[114]  Heba A. Elnemr.  Convolutional Neural Network Architecture for Plant Seedling Classification , 2019 .

[115]  Aravind Krishnaswamy Rangarajan,et al.  Disease Classification in Eggplant Using Pre-trained VGG16 and MSVM , 2020, Scientific Reports.

[116]  K. Swain,et al.  Adoption of an unmanned helicopter for low-altitude remote sensing to estimate yield and total biomass of a rice crop. , 2010 .

[117]  Hong Zhang,et al.  Rice Blast Disease Recognition Using a Deep Convolutional Neural Network , 2019, Scientific Reports.

[118]  Marc Rußwurm,et al.  BreizhCrops: A Satellite Time Series Dataset for Crop Type Identification , 2019, ArXiv.

[119]  Vinay P. Namboodiri,et al.  Deep active learning for object detection , 2018, BMVC.

[120]  Shivaji Bachche,et al.  Deliberation on Design Strategies of Automatic Harvesting Systems: A Survey , 2015, Robotics.

[121]  Wei Guo,et al.  Active learning with point supervision for cost-effective panicle detection in cereal crops , 2020, Plant Methods.

[122]  Adel Hafiane,et al.  V INE DISEASE DETECTION IN UAV MULTISPECTRAL IMAGES WITH DEEP LEARNING SEGMENTATION APPROACH , 2019 .

[123]  Qi Liu,et al.  Visual Tea Leaf Disease Recognition Using a Convolutional Neural Network Model , 2019, Symmetry.

[124]  Zhongzhi Han,et al.  Research on insect pest image detection and recognition based on bio-inspired methods , 2018 .

[125]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[126]  Xianzhang Pan,et al.  Mapping Soil Texture of a Plain Area Using Fuzzy-c-Means Clustering Method Based on Land Surface Diurnal Temperature Difference , 2012 .

[127]  Karthikeyan Natesan Ramamurthy,et al.  Counting and Segmenting Sorghum Heads , 2019, ArXiv.

[128]  De-Cheng Li,et al.  Retrieval and Mapping of Soil Texture Based on Land Surface Diurnal Temperature Range Data from MODIS , 2015, PloS one.

[129]  Ausif Mahmood,et al.  A Framework for Designing the Architectures of Deep Convolutional Neural Networks , 2017, Entropy.

[130]  Mohsen Guizani,et al.  Unmanned Aerial Vehicles (UAVs): A Survey on Civil Applications and Key Research Challenges , 2018, IEEE Access.

[131]  Sang Cheol Kim,et al.  A Robust Deep-Learning-Based Detector for Real-Time Tomato Plant Diseases and Pests Recognition , 2017, Sensors.