Computer vision technology in agricultural automation —A review

Abstract Computer vision is a field that involves making a machine “see”. This technology uses a camera and computer instead of the human eye to identify, track and measure targets for further image processing. With the development of computer vision, such technology has been widely used in the field of agricultural automation and plays a key role in its development. This review systematically summarizes and analyzes the technologies and challenges over the past three years and explores future opportunities and prospects to form the latest reference for researchers. Through the analyses, it is found that the existing technology can help the development of agricultural automation for small field farming to achieve the advantages of low cost, high efficiency and high precision. However, there are still major challenges. First, the technology will continue to expand into new application areas in the future, and there will be more technological issues that need to be overcome. It is essential to build large-scale data sets. Second, with the rapid development of agricultural automation, the demand for professionals will continue to grow. Finally, the robust performance of related technologies in various complex environments will also face challenges. Through analysis and discussion, we believe that in the future, computer vision technology will be combined with intelligent technology such as deep learning technology, be applied to every aspect of agricultural production management based on large-scale datasets, be more widely used to solve the current agricultural problems, and better improve the economic, general and robust performance of agricultural automation systems, thus promoting the development of agricultural automation equipment and systems in a more intelligent direction.

[1]  Nesar Mohammadi Baneh,et al.  Mechatronic components in apple sorting machines with computer vision , 2018, Journal of Food Measurement and Characterization.

[2]  Daoliang Li,et al.  Model predictive control and its application in agriculture: A review , 2018, Comput. Electron. Agric..

[3]  Junho Yeom,et al.  Validation of agronomic UAV and field measurements for tomato varieties , 2019, Comput. Electron. Agric..

[4]  P. K. Langat,et al.  Improving Irrigation Water Use Efficiency: A Review of Advances, Challenges and Opportunities in the Australian Context , 2018, Water.

[5]  Andrea Luvisi,et al.  iPathology: Robotic Applications and Management of Plants and Plant Diseases , 2017 .

[6]  S. Minaei,et al.  Sweet lemon mechanical damage detection using image processing technique and UV radiation , 2018, Journal of Food Measurement and Characterization.

[7]  Haitham Y. Bahlol,et al.  Irrigated pinto bean crop stress and yield assessment using ground based low altitude remote sensing technology , 2019, Information Processing in Agriculture.

[8]  Wei Guo,et al.  High-Throughput Phenotyping of Sorghum Plant Height Using an Unmanned Aerial Vehicle and Its Application to Genomic Prediction Modeling , 2017, Front. Plant Sci..

[9]  Mohammad Saber Iraji,et al.  Comparison between soft computing methods for tomato quality grading using machine vision , 2018, Journal of Food Measurement and Characterization.

[10]  Michael Förster,et al.  UAV data as alternative to field sampling to map woody invasive species based on combined Sentinel-1 and Sentinel-2 data , 2019, Remote Sensing of Environment.

[11]  Ming Yu,et al.  Spatial-Spectral Fusion Based on Conditional Random Fields for the Fine Classification of Crops in UAV-Borne Hyperspectral Remote Sensing Imagery , 2019, Remote. Sens..

[12]  Patrizia Busato,et al.  Machine Learning in Agriculture: A Review , 2018, Sensors.

[13]  Dionysis Bochtis,et al.  Advances in agricultural machinery management: A review , 2014 .

[14]  Zifeng Guo,et al.  A framework for the management of agricultural resources with automated aerial imagery detection , 2019, Comput. Electron. Agric..

[15]  Grazia Tucci,et al.  Multi-Sensor UAV Application for Thermal Analysis on a Dry-Stone Terraced Vineyard in Rural Tuscany Landscape , 2019, ISPRS Int. J. Geo Inf..

[16]  U Andayani,et al.  Image processing analysis of geospatial uav orthophotos for palm oil plantation monitoring , 2018 .

[17]  L. Chen.  Significance of cultivation of new agricultural management subjects for the development goal of modern agriculture: a perspective review , 2018, IOP Conference Series: Earth and Environmental Science.

[18]  Daniel A. Wachspress,et al.  Prediction of Aerial Spray Release from UAVs , 2018 .

[19]  Harish Kumar Sardana,et al.  Image acquisition techniques for assessment of legume quality , 2015 .

[20]  Néstor Lucas Martínez,et al.  A Mission Planning Approach for Precision Farming Systems Based on Multi-Objective Optimization , 2018, Sensors.

[21]  Leanne Lai Hang Chan,et al.  Multi-class fruit detection based on multiple color channels , 2015, 2015 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR).

[22]  Partha Pratim Ray,et al.  Internet of things for smart agriculture: Technologies, practices and future direction , 2017, J. Ambient Intell. Smart Environ..

[23]  Ghulam Muhammad,et al.  Automatic Fruit Classification Using Deep Learning for Industrial Applications , 2019, IEEE Transactions on Industrial Informatics.

[24]  Siza D. Tumbo,et al.  Unmanned aerial vehicle-based remote sensing in monitoring smallholder, heterogeneous crop fields in Tanzania , 2018 .

[25]  Wen Zhang,et al.  A review on weed detection using ground-based machine vision and image processing techniques , 2019, Comput. Electron. Agric..

[26]  Noboru Noguchi,et al.  Development of a positioning system using UAV-based computer vision for an airboat navigation in paddy field , 2019, Comput. Electron. Agric..

[27]  Changki Mo,et al.  Proof-of-concept of a robotic apple harvester , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Guicheng Shi,et al.  General review of intelligent agriculture development in China , 2018, China Agricultural Economic Review.

[29]  Ashutosh Kumar Singh,et al.  Deep Learning for Plant Stress Phenotyping: Trends and Future Perspectives. , 2018, Trends in plant science.

[30]  Duke M. Bulanon,et al.  Machine vision for orchard navigation , 2018, Comput. Ind..

[31]  Giulio Reina,et al.  Agricultural robot for radicchio harvesting , 2006, J. Field Robotics.

[32]  S. J. Miles,et al.  Influence of Mushroom Strains and Population Density on the Performance of a Robotic Harvester , 1997 .

[33]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[34]  Alexander Wendel,et al.  Ground based hyperspectral imaging for extensive mango yield estimation , 2019, Comput. Electron. Agric..

[35]  Ginés García-Mateos,et al.  A fast and accurate expert system for weed identification in potato crops using metaheuristic algorithms , 2018, Comput. Ind..

[36]  Hongzhou Lian,et al.  A far-red-emitting NaMgLaTeO6:Mn4+ phosphor with perovskite structure for indoor plant growth , 2019, Dyes and Pigments.

[37]  T. Gomiero.  Agriculture and degrowth: State of the art and assessment of organic and biotech-based agriculture from a degrowth perspective , 2017, Journal of Cleaner Production.

[38]  Young K. Chang,et al.  Current and future applications of statistical machine learning algorithms for agricultural machine vision systems , 2019, Comput. Electron. Agric..

[39]  J. Kim,et al.  The Study of MP-MAS Utilization to Support Decision-Making for Climate-Smart Agriculture in Rice Farming , 2016 .

[40]  Zhiguo Cao,et al.  In-field automatic observation of wheat heading stage using computer vision , 2016 .

[41]  Raul Morais,et al.  Recent advances in image processing techniques for automated harvesting purposes: A review , 2017, 2017 Intelligent Systems Conference (IntelliSys).

[42]  Fabiana Rodrigues Leta,et al.  Applications of computer vision techniques in the agriculture and food industry: a review , 2012, European Food Research and Technology.

[43]  Ke Wang,et al.  Utilization of Machine Vision to Monitor the Dynamic Responses of Rice Leaf Morphology and Colour to Nitrogen, Phosphorus, and Potassium Deficiencies , 2018, Journal of Spectroscopy.

[44]  Tristan Perez,et al.  Fruit Quantity and Ripeness Estimation Using a Robotic Vision System , 2018, IEEE Robotics and Automation Letters.

[45]  Xiaodong Yang,et al.  Modeling maize above-ground biomass based on machine learning approaches using UAV remote-sensing data , 2019, Plant Methods.

[46]  Raul Morais,et al.  Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry , 2017, Remote. Sens..

[47]  Wenting Han,et al.  Decision Support System for Variable Rate Irrigation Based on UAV Multispectral Remote Sensing , 2019, Sensors.

[48]  K. Kusnierek,et al.  Proof-of-concept robot platform for exploring automated harvesting of sugar snap peas , 2017, Precision Agriculture.

[49]  Reza Ehsani,et al.  A methodology for fresh tomato maturity detection using computer vision , 2018, Comput. Electron. Agric..

[50]  Martin John Oates,et al.  Development of a visual monitoring system for water balance estimation of horticultural crops using low cost cameras , 2017, Comput. Electron. Agric..

[51]  Rubens A. C. Lamparelli,et al.  Mapping skips in sugarcane fields using object-based analysis of unmanned aerial vehicle (UAV) images , 2017, Comput. Electron. Agric..

[52]  Luxon Nhamo,et al.  Improving the Accuracy of Remotely Sensed Irrigated Areas Using Post-Classification Enhancement Through UAV Capability , 2018, Remote. Sens..

[53]  Cyrill Stachniss,et al.  WeedMap: A large-scale semantic weed mapping framework using aerial multispectral imaging and deep neural network for precision farming , 2018, Remote. Sens..

[54]  Lingxian Zhang,et al.  A segmentation method for processing greenhouse vegetable foliar disease symptom images , 2019, Information Processing in Agriculture.

[55]  Hong Sun,et al.  Potato quality grading based on machine vision and 3D shape analysis , 2018, Comput. Electron. Agric..

[56]  Sreekala G. Bajwa,et al.  Detection of soybean aphids in a greenhouse using an image processing technique , 2017, Comput. Electron. Agric..

[57]  F. Danesi,et al.  Ancient wheat and health: a legend or the reality? A review on KAMUT khorasan wheat , 2017, International journal of food sciences and nutrition.

[58]  Yan Li,et al.  Underwater video mosaicing using topology and superpixel-based pairwise stitching , 2019, Expert Syst. Appl..

[59]  Pouria Sadeghi-Tehran,et al.  Automated Method to Determine Two Critical Growth Stages of Wheat: Heading and Flowering , 2017, Front. Plant Sci..

[60]  A. Kamilaris,et al.  A review of the use of convolutional neural networks in agriculture , 2018, The Journal of Agricultural Science.

[61]  Muhammad Kamran,et al.  Towards real-time crops surveillance for disease classification: exploiting parallelism in computer vision , 2017, Comput. Electr. Eng..

[62]  Wang Dong,et al.  Foresight of Disruptive Technologies in Agricultural Engineering , 2018 .

[63]  Giancarlo Troni,et al.  A pattern recognition strategy for visual grape bunch detection in vineyards , 2018, Comput. Electron. Agric..

[64]  Huajian Liu,et al.  A multispectral machine vision system for invertebrate detection on green leaves , 2018, Comput. Electron. Agric..

[65]  Hamid Saeed Khan,et al.  Modern Trends in Hyperspectral Image Analysis: A Review , 2018, IEEE Access.

[66]  David Reiser,et al.  3-D Imaging Systems for Agricultural Applications—A Review , 2016, Sensors.

[67]  Farid Melgani,et al.  Computer vision-based phenotyping for improvement of plant productivity: a machine learning perspective , 2018, GigaScience.

[68]  M. Anwar Hossain,et al.  Botanical Internet of Things: Toward Smart Indoor Farming by Connecting People, Plant, Data and Clouds , 2018, Mob. Networks Appl..

[69]  L. Sass,et al.  PlantSize Offers an Affordable, Non-destructive Method to Measure Plant Size and Color in Vitro , 2018, Front. Plant Sci..

[70]  Deng Limiao,et al.  A Carrot Sorting System Using Machine Vision Technique , 2019, Computer Vision-Based Agriculture Engineering.

[71]  Cyrill Stachniss,et al.  Robust Long-Term Registration of UAV Images of Crop Fields for Precision Agriculture , 2018, IEEE Robotics and Automation Letters.

[72]  C. Daughtry,et al.  What good are unmanned aircraft systems for agricultural remote sensing and precision agriculture? , 2018 .

[73]  Daniel Garbellini Duft,et al.  The potential for RGB images obtained using unmanned aerial vehicle to assess and predict yield in sugarcane fields , 2018 .

[74]  H. A. Hingoliwala,et al.  Smart Farming: Pomegranate Disease Detection Using Image Processing , 2015 .

[75]  Wenting Han,et al.  Estimating Above-Ground Biomass of Maize Using Features Derived from UAV-Based RGB Imagery , 2019, Remote. Sens..

[76]  Peter McCloskey,et al.  A Mobile-Based Deep Learning Model for Cassava Disease Diagnosis , 2019, Front. Plant Sci..

[77]  José M. Bengochea-Guevara,et al.  Comparing UAV-Based Technologies and RGB-D Reconstruction Methods for Plant Height and Biomass Monitoring on Grass Ley , 2019, Sensors.

[78]  Xiaoyu Chen,et al.  Unmanned Aerial Vehicle for Remote Sensing Applications - A Review , 2019, Remote. Sens..

[79]  Zhao Dean,et al.  A nighttime image enhancement method based on Retinex and guided filter for object recognition of apple harvesting robot , 2018 .

[80]  Rafael Rieder,et al.  Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review , 2018, Comput. Electron. Agric..

[81]  Siva Kumar Balasundram,et al.  Research and development in agricultural robotics: a perspective of digital farming. , 2018 .

[82]  Chung-Liang Chang,et al.  Smart Agricultural Machine with a Computer Vision-Based Weeding and Variable-Rate Irrigation Scheme , 2018, Robotics.

[83]  Yao Zhou,et al.  A Vision-Based Counting and Recognition System for Flying Insects in Intelligent Agriculture , 2018, Sensors.

[84]  Viacheslav I. Adamchuk,et al.  Microscope-based computer vision to characterize soil texture and soil organic matter. , 2016 .

[85]  Guangtao Zhai,et al.  Application of Deep Learning Architectures for Accurate and Rapid Detection of Internal Mechanical Damage of Blueberry Using Hyperspectral Transmittance Data , 2018, Sensors.

[86]  Yan Wang,et al.  Cherry recognition in natural environment based on the vision of picking robot , 2017 .

[87]  Jessica Lindblom,et al.  Promoting sustainable intensification in precision agriculture: review of decision support systems development and strategies , 2016, Precision Agriculture.

[88]  José Carlos Barbosa,et al.  Automatic green fruit counting in orange trees using digital images , 2016, Comput. Electron. Agric..

[89]  Hui Cheng,et al.  Automatic delivery and recovery system of Wireless Sensor Networks (WSN) nodes based on UAV for agricultural applications , 2019, Comput. Electron. Agric..

[90]  Zahid Iqbal,et al.  Detection and classification of citrus diseases in agriculture based on optimized weighted segmentation and feature selection , 2018, Comput. Electron. Agric..

[91]  Tao Wang,et al.  SeeFruits: Design and evaluation of a cloud-based ultra-portable NIRS system for sweet cherry quality detection , 2018, Comput. Electron. Agric..

[92]  Emanuel Peres,et al.  UAV-Based Automatic Detection and Monitoring of Chestnut Trees , 2019, Remote. Sens..

[93]  C. Wachenheim,et al.  Technology adoption among farmers in Jilin Province, China , 2018, China Agricultural Economic Review.

[94]  Caio T. Fongaro,et al.  Geospatial Soil Sensing System (GEOS3): A powerful data mining procedure to retrieve soil spectral reflectance from satellite images , 2018, Remote Sensing of Environment.

[95]  Yu Sun,et al.  Automatic Image-Based Plant Disease Severity Estimation Using Deep Learning , 2017, Comput. Intell. Neurosci..

[96]  Kah Phooi Seng,et al.  Big data and machine learning for crop protection , 2018, Comput. Electron. Agric..

[97]  J. Álvarez-Bermejo,et al.  Development of automatic counting system for urediospores of wheat stripe rust based on image processing , 2017 .

[98]  Malik Jahan Khan,et al.  An intelligent mobile application for diagnosis of crop diseases in Pakistan using fuzzy inference system , 2018, Comput. Electron. Agric..

[99]  Mario Castelán,et al.  A contextualized approach for segmentation of foliage in different crop species , 2019, Comput. Electron. Agric..

[100]  Mauro Villarini,et al.  Smart Machines, Remote Sensing, Precision Farming, Processes, Mechatronic, Materials and Policies for Safety and Health Aspects , 2018 .