Autonomous Crop Row Guidance Using Adaptive Multi-ROI in Strawberry Fields

Automated robotic platforms are an important part of precision agriculture solutions for sustainable food production. Agri-robots require robust and accurate guidance systems in order to navigate between crops and to and from their base station. Onboard sensors such as machine vision cameras offer a flexible guidance alternative to more expensive solutions for structured environments such as scanning lidar or RTK-GNSS. The main challenges for visual crop row guidance are the dramatic differences in appearance of crops between farms and throughout the season and the variations in crop spacing and contours of the crop rows. Here we present a visual guidance pipeline for an agri-robot operating in strawberry fields in Norway that is based on semantic segmentation with a convolution neural network (CNN) to segment input RGB images into crop and not-crop (i.e., drivable terrain) regions. To handle the uneven contours of crop rows in Norway’s hilly agricultural regions, we develop a new adaptive multi-ROI method for fitting trajectories to the drivable regions. We test our approach in open-loop trials with a real agri-robot operating in the field and show that our approach compares favourably to other traditional guidance approaches.

[1]  Lars Grimstad,et al.  A Novel Autonomous Robot for Greenhouse Applications , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[2]  R. P. Singh,et al.  Automatic semantic segmentation and classification of remote sensing data for agriculture , 2018, Mathematical Models in Engineering.

[3]  Ivan Vidovic,et al.  Crop row detection by global energy minimization , 2016, Pattern Recognit..

[4]  G. van Straten,et al.  A vision based row detection system for sugar beet , 2005 .

[5]  H. T. Søgaard,et al.  Determination of crop rows by image analysis without segmentation , 2003 .

[6]  Varun Varshney Supervised and unsupervised learning for plant and crop row detection in precision agriculture , 2017 .

[7]  Cyrill Stachniss,et al.  Real-Time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Guoquan Jiang,et al.  Automatic detection of crop rows based on multi-ROIs , 2015, Expert Syst. Appl..

[9]  Pål Johan From,et al.  Design and Evaluation of a Novel Cable-Driven Gripper with Perception Capabilities for Strawberry Picking Robots , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[10]  John F. Reid,et al.  Hough-transform-based vision algorithm for crop row detection of an automated agricultural vehicle , 2005 .

[11]  Wolfram Burgard,et al.  Crop Row Detection on Tiny Plants With the Pattern Hough Transform , 2018, IEEE Robotics and Automation Letters.

[12]  Qin Zhang,et al.  Development of a stereovision sensing system for 3D crop row structure mapping and tractor guidance , 2008 .

[13]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Lars Grimstad,et al.  The Thorvald II Agricultural Robotic System , 2017, Robotics.

[15]  George E. Meyer,et al.  Plant species identification, size, and enumeration using machine vision techniques on near-binary images , 1993, Other Conferences.

[16]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Long Qi,et al.  Fully convolutional network for rice seedling and weed image segmentation at the seedling stage in paddy fields , 2019, PloS one.

[18]  Joe MacInnes,et al.  A Pseudo-Derivative Method for Sliding Window Path Mapping in Robotics-Based Image Processing , 2019 .

[19]  David Ball,et al.  Learning crop models for vision-based guidance of agricultural robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Xavier P. Burgos-Artizzu,et al.  utomatic segmentation of relevant textures in agricultural images , 2010 .

[21]  Gonzalo Pajares,et al.  Automatic detection of curved and straight crop rows from images in maize fields , 2017 .

[22]  Shih-Fang Chen,et al.  Development of Navigation System for Tea Field Machine Using Semantic Segmentation , 2019, IFAC-PapersOnLine.

[23]  G. Meyer,et al.  Verification of color vegetation indices for automated crop imaging applications , 2008 .

[24]  Gonzalo Pajares,et al.  Support Vector Machines for crop/weeds identification in maize fields , 2012, Expert Syst. Appl..

[25]  Tofael Ahamed,et al.  Monitoring of Plant Growth Using Laser Range Finder , 2011 .