Development of a 3D ego-motion estimation system for an autonomous agricultural vehicle

A stereo vision based three-dimensional (3D) ego-motion estimation system was proposed and tested to enable real-time navigation for a full-scale agricultural vehicle. A stereo camera was used to track features in image sequences which were then matched to obtain 3D point clouds that can be used for motion estimation. To overcome the challenges of uneven terrains, relative rotation and translation motions were taken into consideration by registering the point clouds using the iterative closest point algorithm. A smooth-motion constraint was employed to reduce estimation outliers, and a multi-frame estimation strategy was developed to limit estimation failures and error propagations. A series of field tests were conducted on different field surfaces. In a soybean field where the vehicle was driven for 2.5 km following a typical back-and-forth route, the maximum position estimation error was 5.12 m or 0.20%; while on a grass road, where available features were limited, the maximum error was 6.21 m, or 1.61%, for a driving distance of 386 m. To evaluate the estimation of heading angles using multiple frames, the vehicle was driven following a sine-wave route for 100 m; the root-mean-square error (RMSE) of heading angle was 1.43°, which was much lower compared to a RMSE of 6.69°obtained using consecutive frames for estimation. Finally, the vehicle was driven across a 0.24 m and a 0.1 m high bumps and an estimate of pitch angle and roll angle were obtained. An RMSE of 0.30°was obtained for both angles, indicating its feasibility of navigating vehicles on uneven terrains.

[1]  Lasitha Piyathilaka,et al.  Vision-only outdoor localization of two-wheel tractor for autonomous operation in agricultural fields , 2011, 2011 6th International Conference on Industrial and Information Systems.

[2]  Noboru Noguchi,et al.  Agricultural automatic guidance research in North America , 2000 .

[3]  Javier Civera,et al.  1-Point RANSAC for extended Kalman filtering: Application to real-time structure from motion and visual odometry , 2010 .

[4]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[5]  Tofael Ahamed,et al.  Development of the Reconfigurable Data Acquisition Vehicle for Bio-Energy Crop Sensing and Management , 2012 .

[6]  B. Astrand,et al.  Stereo Visual Odometry for Mobile Robots on Uneven Terrain , 2008, Advances in Electrical and Electronics Engineering - IAENG Special Edition of the World Congress on Engineering and Computer Science 2008.

[7]  Roland Siegwart,et al.  Stereo-Based Ego-Motion Estimation Using Pixel Tracking and Iterative Closest Point , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[8]  Ruizhi Chen,et al.  Effect of camera characteristics on the accuracy of a visual gyroscope for indoor pedestrian navigation , 2012, 2012 Ubiquitous Positioning, Indoor Navigation, and Location Based Service (UPINLBS).

[9]  T. F. Burks,et al.  Greenhouse Robot Navigation Using KLT Feature Tracking for Visual Odometry , 2007 .

[10]  Kurt Konolige,et al.  Real-time Localization in Outdoor Environments using Stereo Vision and Inexpensive GPS , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Ezzeddine Zagrouba,et al.  An efficient image-mosaicing method based on multifeature matching , 2009, Machine Vision and Applications.

[12]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[13]  Albert-Jan Baerveldt,et al.  An Agricultural Mobile Robot with Vision-Based Perception for Mechanical Weed Control , 2002, Auton. Robots.

[14]  Qin Zhang,et al.  Stereovision-based lateral offset measurement for vehicle navigation in cultivated stubble fields , 2011 .

[15]  N. D. Tillett,et al.  Ground based sensing systems for autonomous agricultural vehicles , 2000 .

[16]  Clark F. Olson,et al.  Rover navigation using stereo ego-motion , 2003, Robotics Auton. Syst..

[17]  Ignacio Parra,et al.  Robust visual odometry for vehicle localization in urban environments , 2009, Robotica.

[18]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[19]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Hyungsuck Cho,et al.  Visual gyroscope: Integration of visual information with gyroscope for attitude measurement of mobile platform , 2008, 2008 International Conference on Control, Automation and Systems.

[21]  Liang Chen,et al.  A two-dimensional pedestrian navigation solution aided with a visual gyroscope and a visual odometer , 2013, GPS Solutions.

[22]  Gordon Dobie,et al.  Visual odometry and image mosaicing for NDE , 2013 .

[23]  Pau Klein,et al.  San Francisco, California , 2007 .

[24]  Feng Gao,et al.  Development of crop close proximity measurements vehicle for Miscanthus , 2011 .

[25]  Francisco Rovira-Más Recent Innovations in Off-Road Intelligent Vehicles: In-Field Automatic Navigation , 2009 .

[26]  Kurt Konolige,et al.  Visual Odometry Using Sparse Bundle Adjustment on an Autonomous Outdoor Vehicle , 2006, AMS.

[27]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[28]  Liu Ji-lin Harris Corner Detection-based Stereo Visual Odometry , 2007 .

[29]  S. Gan-Mor,et al.  Implement lateral position accuracy under RTK-GPS tractor guidance , 2007 .

[30]  Philippe Martinet,et al.  Automatic Guidance of a Farm Tractor Relying on a Single CP-DGPS , 2002, Auton. Robots.

[31]  Björn Åstrand,et al.  Algorithms for visual odometry in outdoor field environment , 2007 .

[32]  Javier Civera,et al.  1‐Point RANSAC for extended Kalman filtering: Application to real‐time structure from motion and visual odometry , 2010, J. Field Robotics.

[33]  Anastasios I. Mourikis,et al.  High-precision, consistent EKF-based visual-inertial odometry , 2013, Int. J. Robotics Res..

[34]  Hossein Mousazadeh,et al.  A technical review on navigation systems of agricultural autonomous off-road vehicles , 2013 .

[35]  Ignacio Parra,et al.  3D Visual Odometry for Road Vehicles , 2008, J. Intell. Robotic Syst..

[36]  Tom Drummond,et al.  A Single-frame Visual Gyroscope , 2005, BMVC.