Sensor Fusion with Out-of-Sequence Measurements - Localization in an Agricultural Robot Using Visual Odometry

Timing is an important aspect when working with visual odometry (VO). The time of processing and data transfer of image based measurements are significantly longer than IMU, wheel encoders, and GPS measurements. This introduces an arrival latency, causing the VO measurements to require delay fusion, a subcategory of out-of-sequence measurement (OOSM) fusion. In cooperation with Adigo AS this thesis has focused on the Asterix project, where an agricultural robot uses a downward facing camera for visual-inertial odometry to aid in localization. The main focus in this work is with the delay fusion problem. By approaching the OOSM fusion with a Bayesian framework, the theory chapter presents a method of fusing delayed displacement measurements. This can be considered a generalization of the stochastic cloning approach. A byproduct of the investigation is an unscented multiple-point smoother capable of defining fixed-points to be smoothed on demand. Simulations and experiments showed that the OOSM fusion methods worked, but model inconsistencies and inaccuracies in the VO measurements negatively affected the results.

[1]  Niels Kjølstad Poulsen,et al.  Incorporation of time delayed measurements in a discrete-time Kalman filter , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[2]  S. Haykin.  Kalman Filtering and Neural Networks , 2001 .

[3]  Jacques Verron,et al.  A singular evolutive extended Kalman filter for data assimilation in oceanography , 1998 .

[4]  P. Groves.  Principles of GNSS, Inertial, and Multisensor Integrated Navigation Systems, Second Edition , 2013 .

[5]  Stergios I. Roumeliotis,et al.  Observability-based Rules for Designing Consistent EKF SLAM Estimators , 2010, Int. J. Robotics Res..

[6]  Alexander Tallund Klungerbo.  Drop-on-demand i presisjonsjordbruk , 2013 .

[7]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[9]  Rudolph van der Merwe,et al.  Sigma-point kalman filters for probabilistic inference in dynamic state-space models , 2004 .

[10]  Frode Urdal.  Design of a Precision Spray Matrix - Valve Matrix Control for Microspraying in Precision Agriculture , 2014 .

[11]  D. Salmond.  Tracking in Uncertain Environments , 1989 .

[12]  Yakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking: Principles and Techniques , 1995 .

[13]  Ronald P. S. Mahler,et al.  Statistical Multisource-Multitarget Information Fusion , 2007 .

[14]  Øystein Grændsen,et al.  Automatic visual Weed Recognition - Detection and Classification of Weed in Row Cultures combining Machine Vision and Artificial Intelligence , 2014 .

[15]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[16]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[17]  Mandar A. Chitre,et al.  A multi-hypothesis solution to data association for the two-frame SLAM problem , 2015, Int. J. Robotics Res..

[18]  Robin J. Evans,et al.  A fixed-lag smoothing solution to out-of-sequence information fusion problems , 2002, Commun. Inf. Syst..

[19]  Richard A. Brown,et al.  Introduction to random signals and applied kalman filtering (3rd ed , 2012 .

[20]  Stergios I. Roumeliotis,et al.  On the treatment of relative-pose measurements for mobile robot localization , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[21]  Andreas Hegyi,et al.  Dual EKF State and Parameter Estimation in Multi-Class First-Order Traffic Flow Models , 2008 .

[22]  Jonathan P. How,et al.  Nonlinearity in Sensor Fusion: Divergence Issues in EKF, modified truncated SOF, and UKF , 2007 .

[23]  Yaakov Bar-Shalom.  Update with out-of-sequence measurements in tracking: exact solution , 2002 .

[24]  Marcello R. Napolitano,et al.  Comparing Optical Flow Algorithms Using 6-DOF Motion of Real-World Rigid Objects , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[25]  Rudolph van der Merwe,et al.  Sigma-Point Kalman Filters for Integrated Navigation , 2004 .

[26]  Jeffrey K. Uhlmann,et al.  Reduced sigma point filters for the propagation of means and covariances through nonlinear transformations , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[27]  Robin J. Evans,et al.  A Bayesian solution and its approximations to out-of-sequence measurement problems , 2003, Inf. Fusion.

[28]  Gaurav S. Sukhatme,et al.  Circumventing dynamic modeling: evaluation of the error-state Kalman filter applied to mobile robot localization , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[29]  Larry H. Matthies,et al.  Visual odometry on the Mars Exploration Rovers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[30]  Leigh McCue,et al.  Handbook of Marine Craft Hydrodynamics and Motion Control [Bookshelf] , 2016, IEEE Control Systems.

[31]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[32]  Vladislav Gavrilets,et al.  Autonomous aerobatic maneuvering of miniature helicopters , 2003 .

[33]  Larry H. Matthies,et al.  Stereo and IMU assisted visual odometry on an OMAP3530 for small robots , 2011, CVPR 2011 WORKSHOPS.

[34]  Bjarne A. Foss,et al.  Applying the unscented Kalman filter for nonlinear state estimation , 2008 .

[35]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[36]  Simon J. Julier,et al.  Skewed approach to filtering , 1998, Defense, Security, and Sensing.

[37]  Stergios I. Roumeliotis,et al.  SC-KF Mobile Robot Localization: A Stochastic Cloning Kalman Filter for Processing Relative-State Measurements , 2007, IEEE Transactions on Robotics.

[38]  Stergios I. Roumeliotis,et al.  Stochastic cloning: a generalized framework for processing relative state measurements , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[39]  R. Carelli,et al.  Dynamic Modeling and Centralized Formation Control of Mobile Robots , 2006, IECON 2006 - 32nd Annual Conference on IEEE Industrial Electronics.