Real-time optical markerless tracking for augmented reality applications

Augmented reality (AR) technology consists in adding computer-generated information (2D/3D) to a real video sequence in such a manner that the real and virtual objects appear coexisting in the same world. To get a realistic illusion, the real and virtual objects must be properly aligned with respect to each other, which requires a robust real-time tracking strategy—one of the bottlenecks of AR applications. In this paper, we describe the limitations and advantages of different optical tracking technologies, and we present our customized implementation of both recursive tracking and tracking by detection approaches. The second approach requires the implementation of a classifier and we propose the use of a Random Forest classifier. We evaluated both approaches in the context of an AR application for design review. Some conclusions regarding the performance of each approach are given.

[1]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[2]  Vincent Lepetit,et al.  Keypoint recognition using randomized trees , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[4]  Manuel Graña,et al.  Comparative Evaluation of Random Forest and Fern Classifiers for Real-Time Feature Matching , 2008 .

[5]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[6]  Dieter Schmalstieg,et al.  OpenTracker: A flexible software design for three-dimensional interaction , 2005, Virtual Reality.

[7]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[8]  V. Lepetit.  On Computer Vision for Augmented Reality , 2008, 2008 International Symposium on Ubiquitous Virtual Reality.

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Ian D. Reid,et al.  Real-Time SLAM Relocalisation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[12]  David W. Murray,et al.  Real-time localization and mapping with wearable active vision , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[13]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[14]  Didier Stricker,et al.  The MATRIS project: real-time markerless camera tracking for Augmented Reality and broadcast applications , 2007, Journal of Real-Time Image Processing.

[15]  Vincent Lepetit,et al.  Combining edge and texture information for real-time accurate 3D camera tracking , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[16]  Yakup Genc,et al.  Real-Time Feature Matching using Adaptive and Spatially Distributed Classification Trees , 2006, BMVC.

[17]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[18]  Vincent Lepetit,et al.  Point matching as a classification problem for fast and robust object pose estimation , 2004, CVPR 2004.

[19]  Manuel Graña,et al.  Random Forest Classifiers for Real-Time Optical Markerless Tracking , 2016, VISAPP.

[20]  Vincent Lepetit,et al.  Monocular Model-based 3d Tracking of Rigid Objects (Foundations and Trends in Computer Graphics and Vision(R)) , 2005 .

[21]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Vincent Lepetit,et al.  Feature Harvesting for Tracking-by-Detection , 2006, ECCV.