GPU-based multilayer invariant EKF for camera localization

Abstract This paper proposes a GPU-based camera localization approach using an RGBD camera and an inertial sensor. Data from the inertial sensor is input into an Iterative Closest Point (ICP) algorithm as initial information for obtaining the camera localization. German-McClure level-set function is set as the energy function of ICP for the sake of the complicated indoor environment. The covariance of ICP is generated by using a Fisher Information Matrix, and then the localization error of ICP can be quantized based on the covariance. This quantization approach is also extended to a multilayer ICP which operates a coarse to fine framework. An Invariant Extended Kalman Filter (IEKF) approach is employed to fuse the two poses which are obtained from inertial sensor and multilayer ICP respectively to reduce the localization error. The crucial steps of IEKF are implemented by GPU for fast computing massive vertexes. The Experiment results show that the proposed approach can track camera more accurately and perform better real-time performance.

[1]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[3]  Vijay Kumar,et al.  Visual inertial odometry for quadrotors on SE(3) , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[4]  François Goulette,et al.  Invariant EKF Design for Scan Matching-Aided Localization , 2015, IEEE Transactions on Control Systems Technology.

[5]  Matthew Webb,et al.  A topometric system for wide area augmented reality , 2011, Comput. Graph..

[6]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Naokazu Yokoya,et al.  Real-time and accurate extrinsic camera parameter estimation using feature landmark database for augmented reality , 2011, Comput. Graph..

[8]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[9]  Rita Cunha,et al.  A nonlinear position and attitude observer on SE(3) using landmark measurements , 2010, Syst. Control. Lett..

[10]  Veronica Teichrieb,et al.  Incremental structural modeling on sparse visual SLAM , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[11]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[12]  Robert A Gatenby,et al.  Principle of maximum Fisher information from Hardy's axioms applied to statistical systems. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  Ian D. Reid,et al.  Dense Reconstruction Using 3D Object Shape Priors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Philippe Martin,et al.  Invariant Extended Kalman Filter: theory and application to a velocity-aided attitude estimation problem , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[15]  Ian D. Reid,et al.  A Unified Energy Minimization Framework for Model Fitting in Depth , 2012, ECCV Workshops.

[16]  François Goulette,et al.  Accurate 3D maps from depth images and motion sensors via nonlinear Kalman filtering , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Soon-Jo Chung,et al.  Vision‐based Localization and Robot‐centric Mapping in Riverine Environments , 2017, J. Field Robotics.

[18]  Anton H. J. de Ruiter,et al.  SO(3)-constrained Kalman filtering with application to attitude estimation , 2014, 2014 American Control Conference.

[19]  Hirokazu Kato,et al.  SlidAR: A 3D positioning method for SLAM-based handheld augmented reality , 2016, Comput. Graph..

[20]  Matthias Nießner,et al.  Combining Inertial Navigation and ICP for Real-time 3D Surface Reconstruction , 2014, Eurographics.

[21]  Stefan Leutenegger,et al.  ElasticFusion: Real-time dense SLAM and light source estimation , 2016, Int. J. Robotics Res..

[22]  Daniel Mendes,et al.  Design and evaluation of a novel out-of-reach selection technique for VR using iterative refinement , 2017, Comput. Graph..

[23]  Gabe Sibley,et al.  Spline Fusion: A continuous-time representation for visual-inertial fusion with application to rolling shutter cameras , 2013, BMVC.

[24]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[25]  Li Ling,et al.  An Iterated Extended Kalman Filter for 3D mapping via Kinect camera , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Charles K. Toth,et al.  Stereo-inertial Odometry Using Nonlinear Optimization , 2015 .

[27]  Stergios I. Roumeliotis,et al.  A First-Estimates Jacobian EKF for Improving SLAM Consistency , 2009, ISER.

[28]  Andrea Censi,et al.  On achievable accuracy for pose tracking , 2009, 2009 IEEE International Conference on Robotics and Automation.

[29]  Roland Siegwart,et al.  Robust visual inertial odometry using a direct EKF-based approach , 2015, IROS 2015.

[30]  Jong-Hwan Kim,et al.  Extended Kalman Filter Based Mobile Robot Localization in Indoor Fire Environments , 2016 .

[31]  Yuichi Taguchi,et al.  MonoRGBD-SLAM: Simultaneous localization and mapping using both monocular and RGBD cameras , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Gamini Dissanayake,et al.  Convergence and Consistency Analysis for a 3-D Invariant-EKF SLAM , 2017, IEEE Robotics and Automation Letters.

[33]  Olaf Kähler,et al.  Real-Time Large-Scale Dense 3D Reconstruction with Loop Closure , 2016, ECCV.

[34]  John J. Leonard,et al.  Real-time large-scale dense RGB-D SLAM with volumetric fusion , 2014, Int. J. Robotics Res..