Visual tracking for Augmented Reality

Localization of mobile devices is an essential task in Augmented Reality and has therefore been an active research topic for many years. Typically, indoor tracking approaches, such as methods based on infrared or ultra-wide-band, require preparations of the environment and special hardware sensors. Conversely, image feature tracking approaches can provide orientation estimates without special tracking hardware installations. With the advent of mobile devices equipped with sensors such as digital cameras, image based localization gains importance in Augmented Reality. Typically, fiducial marker tracking was considered as a standard image based localization method. We propose the use of natural image feature based tracking methods, which are a generalization of the same principals but do not require the presence of fiducial tracking targets.

[1]  Simon J. Julier,et al.  A tracker alignment framework for augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[2]  Horst Bischof,et al.  From structure-from-motion point clouds to fast location recognition , 2009, CVPR.

[3]  Kosuke Sato,et al.  A wearable mixed reality with an on-board projector , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[4]  Oliver Bimber,et al.  The Virtual Showcase , 2001, IEEE Computer Graphics and Applications.

[5]  Wouter Pasman,et al.  Implementation of an augmented reality system on a PDA , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[6]  Dieter Schmalstieg,et al.  First steps towards handheld augmented reality , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[7]  Nassir Navab,et al.  Fusion of 3D and Appearance Models for Fast Object Detection and Pose Estimation , 2006, ACCV.

[8]  Tom Drummond,et al.  Robust visual tracking for non-instrumental augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[9]  Dieter Schmalstieg,et al.  Pose tracking from natural features on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[10]  Andrew Calway,et al.  Real-Time Camera Tracking Using a Particle Filter , 2005, BMVC.

[11]  Tom Drummond,et al.  Multi-Modal Tracking using Texture Changes , 2004, BMVC.

[12]  Shumin Zhai,et al.  Camera phone based motion sensing: interaction techniques, applications and performance study , 2006, UIST.

[13]  Ronald Azuma,et al.  Improving static and dynamic registration in an optical see-through HMD , 1994, SIGGRAPH.

[14]  Ulrich Neumann,et al.  Fast color fiducial detection and dynamic workspace extension in video see-through self-tracking augmented reality , 1997, Proceedings The Fifth Pacific Conference on Computer Graphics and Applications.

[15]  Hideyuki Tamura,et al.  Robot vision-based registration utilizing bird's-eye view with user's view , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[16]  Frank J. Ferrin.  Survey of helmet tracking technologies , 1991, Medical Imaging.

[17]  Richard Szeliski,et al.  City-Scale Location Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Richard L. Holloway,et al.  Registration errors in augmented reality systems , 1996 .

[19]  Ulrich Neumann,et al.  Multi-ring color fiducial systems for scalable fiducial tracking augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[20]  Dieter Schmalstieg,et al.  Practical calibration procedures for augmented reality , 2000 .

[21]  Fan Xiao,et al.  Display-relative calibration for optical see-through head-mounted displays , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[22]  Ryugo Kijima,et al.  Reflex HMD to compensate lag and correction of derivative deformation , 2002, Proceedings IEEE Virtual Reality 2002.

[23]  Andy Hopper,et al.  Implementing a Sentient Computing System , 2001, Computer.

[24]  Venkataraman Sundareswaran,et al.  Visual servoing-based augmented reality , 1999 .

[25]  Matthias M. Wloka,et al.  Resolving occlusion in augmented reality , 1995, I3D '95.

[26]  Dieter Schmalstieg,et al.  Fast calibration for augmented reality , 1999, VRST '99.

[27]  Tomás Cipra,et al.  Robust Kalman filter and its application in time series analysis , 1991, Kybernetika.

[28]  Bernd Kleinjohann,et al.  Mobile AR4ALL , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[29]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[30]  Thomas P. Caudell,et al.  Calibration of head-mounted displays for augmented reality applications , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[31]  Khoi Nguyen,et al.  Computer-vision-based registration techniques for augmented reality , 1996, Other Conferences.

[32]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[33]  David G. Lowe,et al.  Scene modelling, recognition and tracking with invariant image features , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[34]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[35]  Fan Xiao,et al.  What is the best fiducial? , 2002, The First IEEE International Workshop Agumented Reality Toolkit,.

[36]  Donald B. Gennery,et al.  Visual tracking of known three-dimensional objects , 1992, International Journal of Computer Vision.

[37]  Naokazu Yokoya,et al.  A Stereo Vision-based Mixed Reality System with Natural Feature Point Tracking , 2004 .

[38]  Jurriaan D. Mulder.  Realistic occlusion effects in mirror-based co-located augmented reality systems , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[39]  Ronald Azuma,et al.  Hybrid inertial and vision tracking for augmented reality registration , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[40]  Holger Regenbrecht,et al.  Detecting dynamic occlusion in front of static backgrounds for AR scenes , 2003 .

[41]  Patrick Bouthemy,et al.  Robust real-time visual tracking using a 2D-3D model-based approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[42]  Jun Rekimoto,et al.  ID CAM: a smart camera for scene capturing and ID recognition , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[43]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Eyal de Lara,et al.  Accurate GSM Indoor Localization , 2005, UbiComp.

[45]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[46]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[47]  Jan-Michael Frahm,et al.  From structure-from-motion point clouds to fast location recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.

[49]  Danica Kragic,et al.  Robust Real-Time Visual Tracking: Comparison, Theoretical Analysis and Performance Evaluation , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[50]  Ji Zhou,et al.  Evaluation of calibration procedures for optical see-through head-mounted displays , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[51]  Dieter Schmalstieg,et al.  Robust Incremental Structure from Motion , 2010 .

[52]  Andrew Zisserman,et al.  Robust Object Tracking , 2001 .

[53]  Michael Gervautz,et al.  Occlusion in collaborative augmented environments , 1999, Comput. Graph..

[54]  Dieter Schmalstieg,et al.  3D Tracking Using On-line Keypoint Learning for Mobile Augmented Reality , 2008, CVPR 2008.

[55]  Eric Foxlin,et al.  Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[56]  Martin Wagner,et al.  Herding Sheep: Live System Development for Distributed Augmented Reality , 2003 .

[57]  Tom Drummond,et al.  Sensor fusion and occlusion refinement for tablet-based AR , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[58]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[59]  Dieter Schmalstieg,et al.  Structured visual markers for indoor pathfinding , 2002, The First IEEE International Workshop Agumented Reality Toolkit,.

[60]  Jürgen Gausemeier,et al.  Development of a real time image based object recognition method for mobile AR-devices , 2003, AFRIGRAPH '03.

[61]  Suya You,et al.  Natural feature tracking for extendible robust augmented realities , 1998 .

[62]  Ronald Azuma,et al.  A demonstrated optical tracker with scalable work area for head-mounted display systems , 1992, I3D '92.

[63]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[64]  Supun Samarasekera,et al.  Real-time global localization with a pre-built visual landmark database , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Stefano Soatto,et al.  Scene and Motion Reconstruction from Defocused and Motion-Blurred Images via Anisotropic Diffusion , 2004, ECCV.

[66]  Mark Billinghurst,et al.  An occlusion capable optical see-through head mount display for supporting co-located collaboration , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[67]  Suya You,et al.  Fusion of vision and gyro tracking for robust augmented reality registration , 2001, Proceedings IEEE Virtual Reality 2001.

[68]  Andy Hopper,et al.  The active badge location system , 1992, TOIS.

[69]  Dieter Schmalstieg,et al.  Real-time panoramic mapping and tracking on mobile phones , 2010, 2010 IEEE Virtual Reality Conference (VR).

[70]  J. P. Mellor,et al.  Enhanced Reality Visualization in a Surgical Environment , 1995 .

[71]  Greg Welch,et al.  The HiBall Tracker: high-performance wide-area tracking for virtual and augmented environments , 1999, VRST '99.

[72]  Ulrich Neumann,et al.  Dynamic registration correction in video-based augmented reality systems , 1995, IEEE Computer Graphics and Applications.

[73]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[74]  Tom Drummond,et al.  A Single-frame Visual Gyroscope , 2005, BMVC.

[75]  Marie-Odile Berger.  Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[76]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[77]  Robert M. Haralick,et al.  Analysis and solutions of the three point perspective pose estimation problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[78]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[79]  Oliver Bimber,et al.  Occlusion shadows: using projected light to generate realistic occlusion effects for view-dependent optical see-through displays , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[80]  Ronald Azuma,et al.  Tracking a head-mounted display in a room-sized environment with head-mounted cameras , 1990, Defense, Security, and Sensing.

[81]  Roberto Cipolla,et al.  An Image-Based System for Urban Navigation , 2004, BMVC.

[82]  Naokazu Yokoya,et al.  A stereoscopic video see-through augmented reality system based on real-time vision-based registration , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[83]  Paul Newman,et al.  Detecting Loop Closure with Scene Sequences , 2007, International Journal of Computer Vision.

[84]  Jun Rekimoto.  A Magnifying Glass Approach to Augmented Reality Systems , 1997, Presence Teleoperators Virtual Environ..

[85]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[86]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[87]  David Ingram,et al.  Augmented reality in a wide area sentient environment , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[88]  Greg Welch,et al.  An Introduction to Kalman Filter , 1995, SIGGRAPH 2001.

[89]  Vincent Lepetit,et al.  Combining edge and texture information for real-time accurate 3D camera tracking , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[90]  G. Klinker,et al.  A fast and robust line-based optical tracker for augmented reality applications , 1999 .

[91]  Tom Drummond,et al.  Scalable Monocular SLAM , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[92]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[93]  Tom Drummond,et al.  Tightly integrated sensor fusion for robust visual tracking , 2004, Image Vis. Comput..

[94]  Jannick P. Rolland,et al.  A Survey of Tracking Technologies for Virtual Environments , 2001 .

[95]  Andrew W. Fitzgibbon,et al.  Reliable Fiducial Detection in Natural Scenes , 2004, ECCV.

[96]  Wei Zhang,et al.  Image Based Localization in Urban Environments , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[97]  Paul Smith,et al.  Edge-based motion segmentation , 2002 .

[98]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[99]  Axel Pinz,et al.  The integration of optical and magnetic tracking for multi-user augmented reality , 1999, Comput. Graph..

[100]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[101]  Dieter Schmalstieg,et al.  Wide area localization on mobile phones , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[102]  Marie-Odile Berger,et al.  A two-stage robust statistical method for temporal registration from features of various type , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[103]  Dieter Schmalstieg,et al.  ARToolKit on the PocketPC platform , 2003, 2003 IEEE International Augmented Reality Toolkit Workshop.

[104]  Benjamin Watson,et al.  Using texture maps to correct for optical distortion in head-mounted displays , 1995, Proceedings Virtual Reality Annual International Symposium '95.

[105]  Tom Drummond,et al.  Dynamic measurement clustering to aid real time tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[106]  Dieter Schmalstieg,et al.  Towards Massively Multi-user Augmented Reality on Handheld Devices , 2005, Pervasive.

[107]  Huei-Yung Lin,et al.  Vehicle Speed Detection and Identification from a Single Motion Blurred Image , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[108]  David E. Breen,et al.  Interactive Occlusion and Automatic Object Placement for Augmented Reality , 1996, Comput. Graph. Forum.

[109]  Nassir Navab,et al.  Single point active alignment method (SPAAM) for optical see-through HMD calibration for AR , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[110]  Ioannis M. Rekleitis.  Steerable Filters and Cepstral Analysis for Optical Flow Calculation from a Single Blurred Image , 1996 .

[111]  Bernd Girod,et al.  Outdoors augmented reality on mobile phone using loxel-based visual feature organization , 2008, MIR '08.