Object Detection and Pose Tracking for Augmented Reality: Recent Approaches

This paper introduces recent progress on techniques of object detection and pose tracking with a monocular camera for augmented reality applications. To visually merge a virtual object onto a real scene with geometrical consistency, a camera pose with respect to the scene needs to be computed. For this issue, many approaches have been proposed in the literature. In this paper, we classify and summarize the recent trends of the solutions as a survey.

[1]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[2]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[3]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Tobias Höllerer,et al.  Evaluation of Interest Point Detectors and Feature Descriptors for Visual Tracking , 2011, International Journal of Computer Vision.

[5]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[6]  Russell H. Taylor,et al.  Visual tracking using the sum of conditional variance , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Vincent Lepetit,et al.  Fast Keypoint Recognition Using Random Ferns , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[9]  David W. Murray,et al.  Real-time localization and mapping with wearable active vision , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[10]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[11]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[13]  Didier Stricker,et al.  Adaptable Model-Based Tracking Using Analysis-by-Synthesis Techniques , 2007, CAIP.

[14]  Éric Marchand,et al.  Toward augmenting everything: Detecting and tracking geometrical features on planar objects , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[15]  Yuichi Yoshida,et al.  CARD: Compact And Real-time Descriptors , 2011, 2011 International Conference on Computer Vision.

[16]  Hideo Saito,et al.  Virtually augmenting hundreds of real pictures: An approach based on learning, retrieval, and tracking , 2010, 2010 IEEE Virtual Reality Conference (VR).

[17]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[18]  Andrew Y. C. Nee,et al.  Augmented reality applications in manufacturing: a survey , 2008 .

[19]  Andrew Calway,et al.  Real-Time Camera Tracking Using Known 3D Models and a Particle Filter , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[20]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[21]  David W. Murray,et al.  Full-3D Edge Tracking with a Particle Filter , 2006, BMVC.

[22]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[23]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Hideyuki Tamura,et al.  MR Platform: a basic body on which mixed reality applications are built , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[25]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[26]  Dieter Schmalstieg,et al.  Pose tracking from natural features on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[27]  Hideo Saito,et al.  Random dot markers , 2011, 2011 IEEE Virtual Reality Conference.

[28]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[29]  Ethan Rublee,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[30]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[31]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[32]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[33]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[34]  TurkMatthew,et al.  Evaluation of Interest Point Detectors and Feature Descriptors for Visual Tracking , 2011 .

[35]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[36]  Eric Foxlin,et al.  Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[37]  Guillaume Caron,et al.  Evaluation of Model based Tracking with TrakMark Dataset , 2011 .

[38]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Hiroyuki Yamamoto,et al.  A Fast Initialization Method for Edge-based Registration Using an Inclination Constraint , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[40]  Alois Knoll,et al.  Mutual Information-Based 3D Object Tracking , 2008, International Journal of Computer Vision.

[41]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Tom Drummond,et al.  Going out: robust model-based tracking for outdoor augmented reality , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[43]  Selim Benhimane,et al.  Homography-based 2D Visual Tracking and Servoing , 2007, Int. J. Robotics Res..

[44]  Masakazu Iwamura,et al.  Use of Affine Invariants in Locally Likely Arrangement Hashing for Camera-Based Document Image Retrieval , 2006, Document Analysis Systems.

[45]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[46]  Richard Bowden,et al.  Mutual Information for Lucas-Kanade Tracking (MILK): An Inverse Compositional Formulation , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Dieter Schmalstieg,et al.  ARToolKitPlus for Pose Trackin on Mobile Devices , 2007 .

[48]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[49]  Peter Kontschieder,et al.  Robust planar target tracking and pose estimation from a single concavity , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[50]  Vincent Lepetit,et al.  ESM-Blur: Handling & rendering blur in 3D tracking and augmentation , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[51]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Claude E. Shannon,et al.  The Mathematical Theory of Communication , 1950 .

[53]  Borko Furht,et al.  Handbook of Augmented Reality , 2011 .

[54]  Fumihisa Shibata,et al.  An intermediate report of TrakMark WG - international voluntary activities on establishing benchmark test schemes for AR/MR geometric registration and tracking methods , 2010, ISMAR.

[55]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[56]  Andrea Torsello,et al.  RUNE-Tag: A high accuracy fiducial marker with strong occlusion resilience , 2011, CVPR 2011.

[57]  Nassir Navab,et al.  Benchmarking template-based tracking algorithms , 2010, Virtual Reality.

[58]  Veronica Teichrieb,et al.  A Survey of Online Monocular Markerless Augmented Reality , 2007 .

[59]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Charles V. Stewart,et al.  Robust Parameter Estimation in Computer Vision , 1999, SIAM Rev..

[61]  Dieter Schmalstieg,et al.  Robust and unobtrusive marker tracking on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[62]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[63]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[64]  Gordon Wyeth,et al.  Visual localisation in outdoor industrial building environments , 2008, 2008 IEEE International Conference on Robotics and Automation.

[65]  Vincent Lepetit,et al.  Point-and-shoot for ubiquitous tagging on mobile phones , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[66]  Vincent Lepetit,et al.  Combining edge and texture information for real-time accurate 3D camera tracking , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[67]  Vincent Lepetit,et al.  Texture-less object tracking with online training using an RGB-D camera , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[68]  Éric Marchand,et al.  Accurate real-time tracking using mutual information , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[69]  Éric Marchand,et al.  3D model-based tracking for UAV position control , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[70]  D. W. F. van Krevelen,et al.  A Survey of Augmented Reality Technologies, Applications and Limitations , 2010, Int. J. Virtual Real..

[71]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[72]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[73]  Reto Lindegger.  Handheld Augmented Reality , 2013 .

[74]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[75]  Ramesh Raskar,et al.  Session details: Course 17: Spatial augmented reality: merging real and virtual worlds , 2007, ACM SIGGRAPH 2007 courses.

[76]  Éric Marchand,et al.  Vision-based space autonomous rendezvous: A case study , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[77]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[78]  K. Satoh,et al.  A hybrid and linear registration method utilizing inclination constraint , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[79]  Vincent Lepetit,et al.  Keypoint recognition using randomized trees , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[80]  Darius Burschka,et al.  Adaptive and Generic Corner Detection Based on the Accelerated Segment Test , 2010, ECCV.

[81]  Alvaro Collet,et al.  Making specific features less discriminative to improve point-based 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[82]  Andrew Zisserman,et al.  Robust Object Tracking , 2001 .

[83]  Selim Benhimane,et al.  Gravity-aware handheld Augmented Reality , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[84]  Éric Marchand,et al.  Real-time Hybrid Tracking using Edge and Texture Information , 2007, Int. J. Robotics Res..

[85]  Jun Rekimoto,et al.  Matrix: a realtime object identification and registration method for augmented reality , 1998, Proceedings. 3rd Asia Pacific Computer Human Interaction (Cat. No.98EX110).

[86]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[87]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[88]  Bernd Girod,et al.  Unified Real-Time Tracking and Recognition with Rotation-Invariant Fast Features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[89]  Jihad El-Sana,et al.  Shape Recognition and Pose Estimation for Mobile Augmented Reality , 2009, IEEE Transactions on Visualization and Computer Graphics.

[90]  Geraldo F. Silveira,et al.  Real-time Visual Tracking under Arbitrary Illumination Changes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.