A Deep Learning Driven Algorithmic Pipeline for Autonomous Navigation in Row-Based Crops

Expensive sensors and inefficient algorithmic pipelines significantly affect the overall cost of autonomous machines. However, affordable robotic solutions are essential to practical usage, and their financial impact constitutes a fundamental requirement to employ service robotics in most fields of application. Among all, researchers in the precision agriculture domain strive to devise robust and cost-effective autonomous platforms in order to provide genuinely large-scale competitive solutions. In this article, we present a complete algorithmic pipeline for row-based crops autonomous navigation, specifically designed to cope with low-range sensors and seasonal variations. Firstly, we build on a robust data-driven methodology to generate a viable path for the autonomous machine, covering the full extension of the crop with only the occupancy grid map information of the field. Moreover, our solution leverages on latest advancement of deep learning optimization techniques and synthetic generation of data to provide an affordable solution that efficiently tackles the well-known Global Navigation Satellite System unreliability and degradation due to vegetation growing inside rows. Extensive experimentation and simulations against computer-generated environments and real-world crops demonstrated the robustness and intrinsic generalizability of our methodology that opens the possibility of highly affordable and fully autonomous machines.

[1]  Tao Sun,et al.  High-Accuracy Adaptive Low-Cost Location Sensing Subsystems for Autonomous Rover in Precision Agriculture , 2020, IEEE Open Journal of Industry Applications.

[2]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[3]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[4]  A. K. Deb,et al.  Design and Development of Intelligent Pesticide Spraying System for Agricultural Robot , 2020, HIS.

[5]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Mark Whitty,et al.  GPS-free Localisation and Navigation of an Unmanned Ground Vehicle for Yield Forecasting in a Vineyard , 2014 .

[7]  S. Andrew Gadsden,et al.  An overview of autonomous crop row navigation strategies for unmanned ground vehicles , 2019, Engineering in Agriculture, Environment and Food.

[8]  Lorenzo Comba,et al.  Cost-effective visual odometry system for vehicle motion control in agricultural environments , 2019, Comput. Electron. Agric..

[9]  Thomas Lagkas,et al.  A compilation of UAV applications for precision agriculture , 2020, Comput. Networks.

[10]  Jiri Matas,et al.  Robust Detection of Lines Using the Progressive Probabilistic Hough Transform , 2000, Comput. Vis. Image Underst..

[11]  Marcello Chiaberge,et al.  Local Motion Planner for Autonomous Navigation in Vineyards with a RGB-D Camera-Based Algorithm and Deep Learning Synergy , 2020, Machines.

[12]  Paul A. Beardsley,et al.  An Approach for Semantic Segmentation of Tree-like Vegetation , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[13]  Marcello Chiaberge,et al.  Refining satellite imagery by using UAV imagery for vineyard environment: A CNN Based approach , 2019, 2019 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor).

[14]  K. Sudduth,et al.  Yield estimation in cotton using UAV-based multi-sensor imagery , 2020 .

[15]  Gang Yu,et al.  BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation , 2020, International Journal of Computer Vision.

[16]  Sorin Grigorescu,et al.  A Survey of Deep Learning Techniques for Autonomous Driving , 2020, J. Field Robotics.

[17]  Yubin Lan,et al.  Assessment of rice leaf blast severity using hyperspectral imaging during late vegetative growth , 2020, Australasian Plant Pathology.

[18]  Lorenzo Comba,et al.  Vineyard detection from unmanned aerial systems images , 2015, Comput. Electron. Agric..

[19]  Wolfram Burgard,et al.  Localization for precision navigation in agricultural fields—Beyond crop row following , 2020, J. Field Robotics.

[20]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[21]  Marcello Chiaberge,et al.  DeepWay: a Deep Learning Waypoint Estimator for Global Path Generation , 2020 .

[22]  Olivier Ly,et al.  A Fully Autonomous Robot for Putting Posts for Trellising Vineyard with Centimetric Accuracy , 2015, 2015 IEEE International Conference on Autonomous Robot Systems and Competitions.

[23]  Marcello Chiaberge,et al.  An Adaptive Row Crops Path Generator with Deep Learning Synergy , 2021, 2021 6th Asia-Pacific Conference on Intelligent Robot Systems (ACIRS).

[24]  I. Nevliudov,et al.  Development of the Architecture of the Base Platform Agricultural Robot for Determining the Trajectory Using the Method of Visual Odometry , 2021, 2021 IEEE XVIIth International Conference on the Perspective Technologies and Methods in MEMS Design (MEMSTECH).

[25]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[26]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[27]  Ales Leonardis,et al.  Integrated Commonsense Reasoning and Deep Learning for Transparent Decision Making in Robotics , 2021, SN Comput. Sci..

[28]  Noboru Noguchi,et al.  Performance comparison of single and multi-GNSS receivers under agricultural fields in Korea , 2016 .

[29]  Michael Milford,et al.  Semantics for Robotic Mapping, Perception and Interaction: A Survey , 2021, Found. Trends Robotics.

[30]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Wenqiang Zhang,et al.  Autonomous navigation for a wolfberry picking robot using visual cues and fuzzy control , 2020 .

[32]  Marcello Chiaberge,et al.  Automatic Path Planning for Unmanned Ground Vehicle Using UAV Imagery , 2019, RAAD.

[33]  Carrick Detweiler,et al.  UAV Localization in Row Crops , 2017, J. Field Robotics.

[34]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  H. Jin Kim,et al.  Low-Drift Visual Odometry in Structured Environments by Decoupling Rotational and Translational Motion , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[37]  Diganta Misra,et al.  Mish: A Self Regularized Non-Monotonic Neural Activation Function , 2019, ArXiv.

[38]  Tom Duckett,et al.  Agricultural Robotics: The Future of Robotic Agriculture , 2018, UKRAS White Papers.

[39]  Marcello Chiaberge,et al.  Autonomous Navigation in Vineyards with Deep Learning at the Edge , 2020 .

[40]  Kate Saenko,et al.  Real-Time Semantic Segmentation With Fast Attention , 2020, IEEE Robotics and Automation Letters.

[41]  Steve Furber,et al.  Navigating the Landscape for Real-Time Localization and Mapping for Robotics and Virtual and Augmented Reality , 2018, Proceedings of the IEEE.

[42]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[43]  Carlos Vallespi,et al.  AUTOMATING ORCHARDS: A SYSTEM OF AUTONOMOUS TRACTORS FOR ORCHARD MAINTENANCE , 2012 .

[44]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[45]  Marcello Chiaberge,et al.  Real-Time Apple Detection System Using Embedded Systems With Hardware Accelerators: An Edge AI Application , 2020, IEEE Access.

[46]  Marcello Chiaberge,et al.  Deep Semantic Segmentation at the Edge for Autonomous Navigation in Vineyard Rows , 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[47]  Antonio Barrientos,et al.  Robots in Agriculture: State of Art and Practical Experiences , 2018 .

[48]  Marcello Chiaberge,et al.  A Cost-Effective Person-Following System for Assistive Unmanned Vehicles with Deep Learning at the Edge , 2020, ArXiv.

[49]  Girish Chowdhary,et al.  Learned Visual Navigation for Under-Canopy Agricultural Robots , 2021, Robotics: Science and Systems.

[50]  R. Sparrow,et al.  Robots in agriculture: prospects, impacts, ethics, and policy , 2020, Precision Agriculture.