Resource and Response Aware Path Planning for Long-term Autonomy of Ground Robots in Agriculture

Achieving long-term autonomy for mobile robots operating in real-world unstructured environments such as farms remains a significant challenge. This is made increasingly complex in the presence of moving humans or livestock. These environments require a robot to be adaptive in its immediate plans, accounting for the state of nearby individuals and the response that they might have to the robot’s actions. Additionally, in order to achieve longer-term goals, consideration of the limited on-board resources available to the robot is required, especially for extended missions such as weeding an agricultural field. To achieve efficient long-term autonomy, it is thus crucial to understand the impact that online dynamic updates to an energy efficient offline plan might have on resource usage whilst navigating through crowds or herds. To address these challenges, a hierarchical planning framework is proposed, integrating an online local dynamic path planner with an offline longer-term objective-based planner. This framework acts to achieve long-term autonomy through awareness of both dynamic responses of individuals to a robot’s motion and the limited resources available. This paper details the hierarchical approach and its integration on a robotic platform, including a comprehensive description of the planning framework and associated perception modules. The approach is evaluated in real-world trials on farms, requiring both consideration of limited battery capacity and the presence of nearby moving individuals. These trials additionally demonstrate the ability of the framework to adapt resource use through variation of the local dynamic planner, allowing adaptive behaviour in changing environments. A summary video is available at https://youtu.be/DGVTrYwJ304.

[1]  Kiril Solovey,et al.  The Critical Radius in Sampling-Based Motion Planning , 2018, Robotics: Science and Systems.

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  Jia Pan,et al.  Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios , 2020, Int. J. Robotics Res..

[4]  Y. Charlie Hu,et al.  Deployment of mobile robots with energy and timing constraints , 2006, IEEE Transactions on Robotics.

[5]  Salah Sukkarieh,et al.  Energy Aware Mission Planning for WMRs on Uneven Terrains , 2019, IFAC-PapersOnLine.

[6]  Salah Sukkarieh,et al.  Predicting Responses to a Robot's Future Motion using Generative Recurrent Neural Networks , 2019, ArXiv.

[7]  Nathan D. Wallace,et al.  Experimental Evaluation of a Hierarchical Operating Framework for Ground Robots in Agriculture , 2021, International Symposium on Experimental Robotics.

[8]  Roland Siegwart,et al.  Lighting‐invariant Adaptive Route Following Using Iterative Closest Point Matching , 2015, J. Field Robotics.

[9]  Avital Bechar,et al.  Agricultural robots for field operations. Part 2: Operations and systems , 2017 .

[10]  Rui Caseiro,et al.  High-Speed Tracking with Kernelized Correlation Filters , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Paulo Peixoto,et al.  3D Lidar-based static and moving obstacle detection in driving environments: An approach based on voxels and multi-region ground planes , 2016, Robotics Auton. Syst..

[12]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[13]  Gilbert Laporte,et al.  The orienteering problem with variable profits , 2013, Networks.

[14]  Yin Zhou,et al.  VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Todd Litman,et al.  Autonomous Vehicle Implementation Predictions: Implications for Transport Planning , 2015 .

[16]  Andreas Krause,et al.  Unfreezing the robot: Navigation in dense, interacting crowds , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[18]  Salah Sukkarieh,et al.  A Hierarchical Framework for Long-term and Robust Deployment of Field Ground Robots in Large-Scale Farming , 2020, 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE).

[19]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[20]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Salah Sukkarieh,et al.  Path Planning in Dynamic Environments using Generative RNNs and Monte Carlo Tree Search , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Pratap Tokekar,et al.  Energy-optimal trajectory planning for car-like robots , 2014, Auton. Robots.

[23]  Michael Bosse,et al.  Driving on Point Clouds: Motion Planning, Trajectory Optimization, and Terrain Assessment in Generic Nonplanar Environments , 2017, J. Field Robotics.

[24]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[25]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Andrea Gasparri,et al.  A Navigation Architecture for Ackermann Vehicles in Precision Farming , 2020, IEEE Robotics and Automation Letters.

[27]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[28]  Stewart Worrall,et al.  Naturalistic Driver Intention and Path Prediction Using Recurrent Neural Networks , 2018, IEEE Transactions on Intelligent Transportation Systems.

[29]  E. Pires,et al.  Path Planning for ground robots in agriculture: a short review , 2020, 2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[30]  Marco Pavone,et al.  The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[31]  Fei Gao,et al.  Teach-Repeat-Replan: A Complete and Robust System for Aggressive Flight in Complex Environments , 2019, IEEE Transactions on Robotics.

[32]  Martin Saska,et al.  Dubins orienteering problem with neighborhoods , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[33]  Stefan Becker,et al.  RED: A Simple but Effective Baseline Predictor for the TrajNet Benchmark , 2018, ECCV Workshops.

[34]  Christos G. Cassandras,et al.  Optimal Routing of Energy-Aware Vehicles in Transportation Networks With Inhomogeneous Charging Nodes , 2018, IEEE Transactions on Intelligent Transportation Systems.

[35]  Zbigniew Michalewicz,et al.  Path Planning in Dynamic Environments , 2005, Innovations in Robot Mobility and Control.

[36]  Yangsheng Xu,et al.  Energy management for four-wheel independent driving vehicle , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[38]  Salah Sukkarieh,et al.  Experimental Validation of Structured Receding Horizon Estimation and Control for Mobile Ground Robot Slip Compensation , 2019, FSR.

[39]  Keith Redmill,et al.  Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[40]  Volkan Isler,et al.  Coverage Path Planning Under the Energy Constraint , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Leonidas J. Guibas,et al.  Frustum PointNets for 3D Object Detection from RGB-D Data , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Richard J. Lorentz,et al.  Using evaluation functions in Monte-Carlo Tree Search , 2016, Theor. Comput. Sci..

[43]  Lukas Rummelhard,et al.  Ground estimation and point cloud segmentation using SpatioTemporal Conditional Random Field , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[44]  Salah Sukkarieh,et al.  Receding horizon estimation and control with structured noise blocking for mobile robot slip compensation , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[45]  Dinesh Manocha,et al.  Reciprocal n-Body Collision Avoidance , 2011, ISRR.

[46]  Mac Schwager,et al.  Correlated Orienteering Problem and its application to informative path planning for persistent monitoring tasks , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Stewart Worrall,et al.  Automatic extrinsic calibration between a camera and a 3D Lidar using 3D point and plane correspondences , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[48]  Alexandre Alahi,et al.  Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[49]  Salah Sukkarieh,et al.  Motion Cost Characterisation of an Omnidirectional WMR on Uneven Terrains , 2019, IFAC-PapersOnLine.

[50]  Tin Lun Lam,et al.  OceanVoy: A Hybrid Energy Planning System for Autonomous Sailboat , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[51]  Greg Mori,et al.  Relational Graph Learning for Crowd Navigation , 2019, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[52]  Salah Sukkarieh,et al.  The orienteering Problem with Replenishment , 2020, 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE).

[53]  Rafia Inam,et al.  Risk Assessment for Human-Robot Collaboration in an automated warehouse scenario , 2018, 2018 IEEE 23rd International Conference on Emerging Technologies and Factory Automation (ETFA).