Pose Estimation of a Mobile Robot Based on Fusion of IMU Data and Vision Data Using an Extended Kalman Filter

Using a single sensor to determine the pose estimation of a device cannot give accurate results. This paper presents a fusion of an inertial sensor of six degrees of freedom (6-DoF) which comprises the 3-axis of an accelerometer and the 3-axis of a gyroscope, and a vision to determine a low-cost and accurate position for an autonomous mobile robot. For vision, a monocular vision-based object detection algorithm speeded-up robust feature (SURF) and random sample consensus (RANSAC) algorithms were integrated and used to recognize a sample object in several images taken. As against the conventional method that depend on point-tracking, RANSAC uses an iterative method to estimate the parameters of a mathematical model from a set of captured data which contains outliers. With SURF and RANSAC, improved accuracy is certain; this is because of their ability to find interest points (features) under different viewing conditions using a Hessain matrix. This approach is proposed because of its simple implementation, low cost, and improved accuracy. With an extended Kalman filter (EKF), data from inertial sensors and a camera were fused to estimate the position and orientation of the mobile robot. All these sensors were mounted on the mobile robot to obtain an accurate localization. An indoor experiment was carried out to validate and evaluate the performance. Experimental results show that the proposed method is fast in computation, reliable and robust, and can be considered for practical applications. The performance of the experiments was verified by the ground truth data and root mean square errors (RMSEs).

[1]  B. J. Silva,et al.  Machine-to-Machine: Possible applications in industrial networks , 2013, 2013 IEEE International Conference on Industrial Technology (ICIT).

[2]  Yongtian Wang,et al.  Fusion of inertial and vision data for accurate tracking , 2012, International Conference on Machine Vision.

[3]  Kwangsoo Kim,et al.  High performance vision tracking system for mobile robot using sensor data fusion with Kalman filter , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Tianmiao Wang,et al.  Monocular vision and IMU based navigation for a small unmanned helicopter , 2012, 2012 7th IEEE Conference on Industrial Electronics and Applications (ICIEA).

[5]  Pieter Jonker,et al.  Real-time computer vision system for mobile robot , 2001, SPIE Optics East.

[6]  Naser El-Sheimy,et al.  A Standard Testing and Calibration Procedure for Low Cost MEMS Inertial Sensors and Units , 2008, Journal of Navigation.

[7]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[8]  Wolfram Burgard,et al.  Particle Filters for Mobile Robot Localization , 2001, Sequential Monte Carlo Methods in Practice.

[9]  Javier Ruiz-del-Solar,et al.  Object recognition using local invariant features for robotic applications: A survey , 2016, Pattern Recognit..

[10]  Luis Merino,et al.  Unmanned Aerial Vehicle Localization Based on Monocular Vision and Online Mosaicking , 2009, J. Intell. Robotic Syst..

[11]  Suya You,et al.  Fusion of vision and gyro tracking for robust augmented reality registration , 2001, Proceedings IEEE Virtual Reality 2001.

[12]  Adaptive-frame-rate monocular vision and IMU fusion for robust indoor positioning , 2013, 2013 IEEE International Conference on Robotics and Automation.

[13]  Huosheng Hu,et al.  Integration of Vision and Inertial Sensors for 3D Arm Motion Tracking in Home-based Rehabilitation , 2007, Int. J. Robotics Res..

[14]  Yi-Ping Hung,et al.  RANSAC-Based DARCES: A New Approach to Fast Automatic Registration of Partially Overlapping Range Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Wolfram Burgard,et al.  On the position accuracy of mobile robot localization based on particle filters combined with scan matching , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Gerhard P. Hancke,et al.  Air Quality Monitoring System Based on ISO/IEC/IEEE 21451 Standards , 2016, IEEE Sensors Journal.

[17]  Gerhard P. Hancke,et al.  Rapid prototyping of a wireless sensor network gateway for the internet of things using off-the-shelf components , 2015, 2015 IEEE International Conference on Industrial Technology (ICIT).

[18]  Dong-Il Cho,et al.  A new localization method for mobile robot by data fusion of vision sensor data and motion sensor data , 2012, 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[19]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[20]  Vincent Lepetit,et al.  Fast Keypoint Recognition in Ten Lines of Code , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Jeff Bird,et al.  Indoor navigation with foot-mounted strapdown inertial navigation and magnetic sensors [Emerging Opportunities for Localization and Tracking] , 2011, IEEE Wireless Communications.

[22]  Juan Li,et al.  A novel system for object pose estimation using fused vision and inertial data , 2017, Inf. Fusion.

[23]  Javeria Farooq.  Object detection and identification using SURF and BoW model , 2016, 2016 International Conference on Computing, Electronic and Electrical Engineering (ICE Cube).

[24]  James Diebel,et al.  Representing Attitude : Euler Angles , Unit Quaternions , and Rotation Vectors , 2006 .

[25]  Rehman Ullah Khan,et al.  Objects Tracking from Natural Features in Mobile Augmented Reality , 2013 .

[26]  State of the art and future developments of the Augmented Reality for measurement applications , 2014 .

[27]  Yakup Genc,et al.  Marker-less tracking for AR: a learning-based approach , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[28]  Angelo M. Sabatini,et al.  Extended Kalman Filter-Based Methods for Pose Estimation Using Visual, Inertial and Magnetic Sensors: Comparative Analysis and Performance Evaluation , 2013, Sensors.

[29]  Prashanth Swamy,et al.  An Improved Tracking using IMU and Vision Fusion for Mobile Augmented Reality Applications , 2014, ArXiv.

[30]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[31]  Zhongliang Jing,et al.  Homography estimation from planar contours in image sequence , 2010 .

[32]  P. Swarnalatha,et al.  Detection of Features to Track Objects and Segmentation Using GrabCut for Application in Marker-less Augmented Reality , 2015 .

[33]  Gerhard P. Hancke,et al.  Using NFC-enabled phones for remote data acquisition and digital control , 2011, IEEE Africon '11.

[34]  C Tomasi,et al.  Shape and motion from image streams: a factorization method. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Shang-Hong Lai,et al.  A consensus sampling technique for fast and robust model fitting , 2009, Pattern Recognit..

[36]  Zheyao Wang,et al.  Motion Measurement Using Inertial Sensors, Ultrasonic Sensors, and Magnetometers With Extended Kalman Filter for Data Fusion , 2012, IEEE Sensors Journal.

[37]  Didier Stricker,et al.  Advanced tracking through efficient image processing and visual-inertial sensor fusion , 2008, 2008 IEEE Virtual Reality Conference.

[38]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[39]  Gang Zhang,et al.  Mobile Robot Localization Based on Extended Kalman Filter , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[40]  F. N. Sibai,et al.  Evaluation of indoor mobile robot localization techniques , 2012, 2012 International Conference on Computer Systems and Industrial Informatics.

[41]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[42]  A. Tanju Erdem,et al.  Fusing Inertial Sensor Data in an Extended Kalman Filter for 3D Camera Tracking , 2015, IEEE Transactions on Image Processing.

[43]  Larry S. Davis,et al.  Model-Based Object Pose in 25 Lines of Code , 1992, ECCV.

[44]  Michael R. M. Jenkin,et al.  Inertial Sensors, GPS, and Odometry , 2008, Springer Handbook of Robotics.

[45]  Naser El-Sheimy,et al.  A Novel Real-Time Reference Key Frame Scan Matching Method , 2017, Sensors.

[46]  Gerhard P. Hancke,et al.  An Energy-Efficient Smart Comfort Sensing System Based on the IEEE 1451 Standard for Green Buildings , 2014, IEEE Sensors Journal.

[47]  David Hernández-López,et al.  A robust and hierarchical approach for the automatic co-registration of intensity and visible images , 2012 .

[48]  Fuqiang Zhou,et al.  Accurate and robust estimation of camera parameters using RANSAC , 2013 .

[49]  Ronald Azuma,et al.  Hybrid inertial and vision tracking for augmented reality registration , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[50]  Mary Alatise,et al.  Pose estimation of a mobile robot using monocular vision and inertial sensors data , 2017, 2017 IEEE AFRICON.

[51]  Nicholas Gans,et al.  Localization through fusion of discrete and continuous epipolar geometry with wheel and IMU odometry , 2011, Proceedings of the 2011 American Control Conference.

[52]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[53]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[54]  Gerhard P. Hancke,et al.  Experimental Link Quality Characterization of Wireless Sensor Networks for Underground Monitoring , 2015, IEEE Transactions on Industrial Informatics.

[55]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[56]  Thomas B. Schön,et al.  Modeling and Calibration of Inertial and Vision Sensors , 2010, Int. J. Robotics Res..

[57]  Christophe Macabiau,et al.  Review and classification of vision-based localisation techniques in unknown environments , 2014 .

[58]  T. Voirin,et al.  Visual-inertial navigation for pinpoint planetary landing using scale-based landmark matching , 2016, Robotics Auton. Syst..

[59]  Ji Hoon Joung,et al.  What does ground tell us? Monocular visual odometry under planar motion constraint , 2011, 2011 11th International Conference on Control, Automation and Systems.

[60]  Zhaokun Li,et al.  A new robust 2D camera calibration method using RANSAC , 2015 .

[61]  Hubert Roth,et al.  Data Fusion of Stereo Vision and Gyroscope for Estimation of Indoor Mobile Robot Orientation , 2012, CESCIT.

[62]  Gerhard P. Hancke,et al.  Multiple Region of Interest Coverage in Camera Sensor Networks for Tele-Intensive Care Units , 2016, IEEE Transactions on Industrial Informatics.

[63]  Liqiang Feng,et al.  Navigating Mobile Robots: Systems and Techniques , 1996 .

[64]  Éric Marchand,et al.  Pose Estimation for Augmented Reality: A Hands-On Survey , 2016, IEEE Transactions on Visualization and Computer Graphics.

[65]  Hobart R. Everett,et al.  Mobile Robot Positioning - Sensors and Techniques , 1997 .

[66]  Peng Chen,et al.  An improved augmented reality system based on AndAR , 2016, J. Vis. Commun. Image Represent..

[67]  Jonas Fredriksson,et al.  Reliable Vehicle Pose Estimation Using Vision and a Single-Track Model , 2014, IEEE Transactions on Intelligent Transportation Systems.

[68]  Nauman Aslam,et al.  Dynamic Fuzzy-Logic Based Path Planning for Mobility-Assisted Localization in Wireless Sensor Networks , 2017, Sensors.

[69]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[70]  Gaurav S. Sukhatme,et al.  Robust localization using relative and absolute position estimates , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[71]  Raja Ariffin Raja Ghazilla,et al.  Reviews on Various Inertial Measurement Unit (IMU) Sensor Applications , 2013, SiPS 2013.

[72]  Vincent Lepetit,et al.  Combining Geometric and Appearance Priors for Robust Homography Estimation , 2010, ECCV.

[73]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .