Terrain classification using ToF sensors for the enhancement of agricultural machinery traversability

Abstract Ground properties influence various aspects of mobile machinery navigation including localization, mobility status or task execution. Excessive slipping, skidding or trapping situations can compromise the vehicle itself or other elements in the workspace. Thus, detecting the soil surface characteristics is an important issue for performing different activities in an efficient, safe and satisfactory manner. In agricultural applications, this point is specially important since activities such as seeding, fertilizing, or ploughing are carried on within off-road landscapes which contain a diversity of terrains that modify the navigation behaviour of the vehicle. Thus, the machinery requires a cognitive capability to understand the surrounding terrain type or its characteristics in order to take the proper guidance or control actions. This work is focused on the soil surface classification by implementing a visual system capable to distinguish between five usual types of off-road terrains. Computer vision and machine learning techniques are applied to characterize the texture and color of images acquired with a Microsoft Kinect V2 sensor. In a first stage, development tests showed that only infra-red and RGB streams are useful to obtain satisfactory accuracy rates (above 90%). The second stage included field trials with the sensor mounted on a mobile robot driving through various agricultural landscapes. These scenarios did not present illumination restrictions nor ideal driving roads; hence, conditions can resemble real agricultural operations. In such circumstances, the proposed approach showed robustness and reliability, obtaining an average of 85.20% of successful classifications when tested along 17 trials within agricultural landscapes.

[1]  Karl Iagnemma,et al.  Self‐supervised terrain classification for planetary surface exploration rovers , 2012, J. Field Robotics.

[2]  Alexandre Escolà,et al.  Discriminating Crop, Weeds and Soil Surface with a Terrestrial LIDAR Sensor , 2013, Sensors.

[3]  Theunis R. Botha,et al.  Digital image correlation techniques for measuring tyre–road interface parameters: Part 2 – Longitudinal tyre slip ratio measurement , 2015 .

[4]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[5]  Giulio Reina,et al.  Self-learning classification of radar features for scene understanding , 2012, Robotics Auton. Syst..

[6]  H. Macher,et al.  FIRST EXPERIENCES WITH KINECT V2 SENSOR FOR CLOSE RANGE 3D MODELLING , 2015 .

[7]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2005, International Journal of Computer Vision.

[8]  David Gorsich,et al.  A technical survey on Terramechanics models for tire–terrain interaction used in modeling and simulation of wheeled vehicles , 2015 .

[9]  Masahiro Ono,et al.  Risk-aware planetary rover operation: Autonomous terrain classification and path planning , 2015, 2015 IEEE Aerospace Conference.

[10]  Kaspar Althoefer,et al.  Track–terrain modelling and traversability prediction for tracked vehicles on soft terrain , 2010 .

[11]  Jangmyung Lee,et al.  In Situ 3D Segmentation of Individual Plant Leaves Using a RGB-D Camera for Agricultural Automation , 2015, Sensors.

[12]  J. R. Rosell-Polo,et al.  Advances in Structured Light Sensors Applications in Precision Agriculture and Livestock Farming , 2015 .

[13]  Jihong Lee,et al.  Terrain Feature Extraction and Classification for Mobile Robots Utilizing Contact Sensors on Rough Terrain , 2012 .

[14]  Pietro Perona,et al.  Learning and prediction of slip from visual information , 2007, J. Field Robotics.

[15]  Giulio Reina,et al.  Ambient awareness for agricultural robotic vehicles , 2016, ArXiv.

[16]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[17]  F. Marinello,et al.  Application of the Kinect sensor for dynamic soil surface characterization , 2015, Precision Agriculture.

[18]  Roberto Manduchi,et al.  Obstacle Detection and Terrain Classification for Autonomous Off-Road Navigation , 2005, Auton. Robots.

[19]  Zongquan Deng,et al.  Longitudinal skid model for wheels of planetary exploration rovers based on terramechanics , 2013 .

[20]  Miguel Torres-Torriti,et al.  Probabilistic approaches for self-tuning path tracking controllers using prior knowledge of the terrain , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Tony E Grift,et al.  Variable field-of-view machine vision based row guidance of an agricultural robot , 2012 .

[22]  James T Schwiegerling,et al.  Field guide to visual and ophthalmic optics , 2004 .

[23]  Andrew Zisserman,et al.  A Statistical Approach to Material Classification Using Image Patch Exemplars , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Karl Iagnemma,et al.  Classification-based wheel slip detection and detector fusion for mobile robots on outdoor terrain , 2009, Auton. Robots.

[25]  Thierry Peynot,et al.  A near-to-far non-parametric learning approach for estimating traversability in deformable terrain , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Alexandre Escolà,et al.  Classifying Agricultural Terrain for Machinery Traversability Purposes , 2016 .

[27]  Lihua Xie,et al.  Comparison of different approaches to visual terrain classification for outdoor mobile robots , 2014, Pattern Recognit. Lett..

[28]  Juan Carlos Fernandez-Diaz,et al.  Characterization of surface roughness of bare agricultural soils using LiDAR , 2010 .

[29]  D. Michel.  System of Terrain Analysis, Energy Estimation and Path Planning for Planetary Exploration by Robot Teams , 2012 .

[30]  Karl Iagnemma,et al.  Vibration-based terrain classification for planetary exploration rovers , 2005, IEEE Transactions on Robotics.

[31]  Gary Witus,et al.  Terrain characterization and classification with a mobile robot , 2006, J. Field Robotics.

[32]  Dietrich Paulus,et al.  Probabilistic terrain classification in unstructured environments , 2013, Robotics Auton. Syst..

[33]  David Ball,et al.  Vision‐based Obstacle Detection and Navigation for an Agricultural Robot , 2016, J. Field Robotics.

[34]  Omowunmi Falola,et al.  Drivable region detection for autonomous robots applied to South African underground mining , 2012 .

[35]  Takashi Kubota,et al.  Autonomous Terrain Classification With Co- and Self-Training Approach , 2016, IEEE Robotics and Automation Letters.

[36]  Karl Iagnemma,et al.  Predictable mobility , 2009, IEEE Robotics & Automation Magazine.

[37]  Gregory Broten,et al.  Probabilistic Obstacle Detection Using 2 1/2 D Terrain Maps , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[38]  Andrew Zisserman,et al.  Classifying Images of Materials: Achieving Viewpoint and Illumination Independence , 2002, ECCV.

[39]  Theunis R. Botha,et al.  Digital image correlation techniques for measuring tyre-road interface parameters : Part 1 - Side-slip angle measurement on rough terrain , 2015 .

[40]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..