Instant Motion Tracking and Its Applications to Augmented Reality

Augmented Reality (AR) brings immersive experiences to users. With recent advances in computer vision and mobile computing, AR has scaled across platforms, and has increased adoption in major products. One of the key challenges in enabling AR features is proper anchoring of the virtual content to the real world, a process referred to as tracking. In this paper, we present a system for motion tracking, which is capable of robustly tracking planar targets and performing relative-scale 6DoF tracking without calibration. Our system runs in real-time on mobile phones and has been deployed in multiple major products on hundreds of millions of devices.

[1]  Tobias Höllerer,et al.  Live tracking and mapping from both general and rotation-only camera motion , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[2]  Daniel Wagner,et al.  User friendly SLAM initialization , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[3]  Dieter Schmalstieg,et al.  Real-time panoramic mapping and tracking on mobile phones , 2010, 2010 IEEE Virtual Reality Conference (VR).

[4]  Selim Benhimane,et al.  Real-time image-based tracking of planes using efficient second-order minimization , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[5]  Vincent Lepetit,et al.  Instant Outdoor Localization and SLAM Initialization from 2.5D Maps , 2015, IEEE Transactions on Visualization and Computer Graphics.

[6]  Yang Li,et al.  YES NO Cartesian Update Update Feature Extraction Feature Extraction Phase Correlation Resample Min Eq . 3 ? Fourier spaceLog-Polar Cross Correlation Model Fourier space Model Sample Sample , 2018 .

[7]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Russell H. Taylor,et al.  Visual tracking using the sum of conditional variance , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Adrian David Cheok,et al.  Augmented Reality Camera Tracking with Homographies , 2002, IEEE Computer Graphics and Applications.

[10]  Frank Dellaert,et al.  Initialization techniques for 3D SLAM: A survey on rotation estimation and its use in pose graph optimization , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Ping Wang,et al.  Robust visual tracking for planar objects using gradient orientation pyramid , 2019, J. Electronic Imaging.

[12]  Yifan Wu,et al.  Planar Object Tracking in the Wild: A Benchmark , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Roland Siegwart,et al.  Maplab: An Open Framework for Research in Visual-Inertial Mapping and Localization , 2017, IEEE Robotics and Automation Letters.

[14]  Gerhard Reitmayr,et al.  Homography-based planar mapping and tracking for mobile phones , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[15]  Michael Bosse,et al.  Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization , 2015, Robotics: Science and Systems.

[16]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[17]  Javier Civera,et al.  Visual-Inertial SLAM Initialization: A General Linear Formulation and a Gravity-Observing Non-Linear Optimization , 2018, 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[18]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.