Visual Tracking for Augmented Reality

In Augmented Reality applications, the real environment is annotated or enhanced with computer-generated graphics. These graphics must be exactly registered to real objects in the scene and this requires AR systems to track a user's viewpoint. This book shows that visual tracking with inexpensive cameras (such as those now often built into mobile computing devices) can be sufficiently robust and accurate for AR applications. Visual tracking has previously been applied to AR, however this has used artificial markers placed in the scene; this is undesirable. This thesis explores edge-based tracking as an efficient and accurate alternative, and explored methods to improve the robustness of edge-based tracking against rapid user motions. This thesis also shows how CAD models of the scene can be used to correctly clip virtual graphics, improving the perceived realism of the scene.

[1]  Venkataraman Sundareswaran,et al.  Visual servoing-based augmented reality , 1999 .

[2]  Vincent Lepetit,et al.  Handling occlusion in augmented reality systems: a semi-automatic method , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[3]  Naokazu Yokoya,et al.  A Stereo Vision-based Mixed Reality System with Natural Feature Point Tracking , 2004 .

[4]  Dieter Schmalstieg,et al.  Fast calibration for augmented reality , 1999, VRST '99.

[5]  G. Klinker,et al.  A fast and robust line-based optical tracker for augmented reality applications , 1999 .

[6]  Andrew Zisserman,et al.  Robust Object Tracking , 2001 .

[7]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[8]  Jannick P. Rolland,et al.  A Survey of Tracking Technologies for Virtual Environments , 2001 .

[9]  Naokazu Yokoya,et al.  A stereoscopic video see-through augmented reality system based on real-time vision-based registration , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[10]  Mark Billinghurst,et al.  An occlusion capable optical see-through head mount display for supporting co-located collaboration , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[11]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[12]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Oliver Bimber,et al.  The virtual showcase , 2005, SIGGRAPH Courses.

[14]  Martin Wagner,et al.  Herding Sheep: Live System Development for Distributed Augmented Reality , 2003 .

[15]  David Ingram,et al.  Augmented reality in a wide area sentient environment , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[16]  Tom Drummond,et al.  A Single-frame Visual Gyroscope , 2005, BMVC.

[17]  Marie-Odile Berger.  Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Jannick P. Rolland,et al.  A compact optical see-through head-worn display with occlusion support , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[19]  Marie-Odile Berger,et al.  A two-stage robust statistical method for temporal registration from features of various type , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[20]  Éric Marchand,et al.  Virtual Visual Servoing: a framework for real‐time augmented reality , 2002, Comput. Graph. Forum.

[21]  Benjamin Watson,et al.  Using texture maps to correct for optical distortion in head-mounted displays , 1995, Proceedings Virtual Reality Annual International Symposium '95.

[22]  Tom Drummond,et al.  Tightly integrated sensor fusion for robust visual tracking , 2004, Image Vis. Comput..

[23]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[24]  Patrick Bouthemy,et al.  Robust real-time visual tracking using a 2D-3D model-based approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  Wei Zhu,et al.  Personalized In-store E-Commerce with the PromoPad: an Augmented Reality Shopping Assistant , 2004 .

[26]  Tom Drummond,et al.  Sensor fusion and occlusion refinement for tablet-based AR , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[27]  F. Hampel.  The Influence Curve and Its Role in Robust Estimation , 1974 .

[28]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[29]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[30]  Jürgen Gausemeier,et al.  Development of a real time image based object recognition method for mobile AR-devices , 2003, AFRIGRAPH '03.

[31]  Pascal Fua,et al.  Texture Boundary Detection for Real-Time Tracking , 2004, ECCV.

[32]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[33]  Wouter Pasman,et al.  Implementation of an augmented reality system on a PDA , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[34]  Tom Drummond,et al.  Multi-Modal Tracking using Texture Changes , 2004, BMVC.

[35]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[36]  Ulrich Neumann,et al.  Fast color fiducial detection and dynamic workspace extension in video see-through self-tracking augmented reality , 1997, Proceedings The Fifth Pacific Conference on Computer Graphics and Applications.

[37]  Frank Chongwoo Park,et al.  Robot sensor calibration: solving AX=XB on the Euclidean group , 1994, IEEE Trans. Robotics Autom..

[38]  Chris Harris,et al.  Tracking with rigid models , 1993 .

[39]  Richard L. Holloway,et al.  Registration errors in augmented reality systems , 1996 .

[40]  Ulrich Neumann,et al.  Multi-ring color fiducial systems for scalable fiducial tracking augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[41]  Dieter Schmalstieg,et al.  Practical calibration procedures for augmented reality , 2000 .

[42]  Ryugo Kijima,et al.  Reflex HMD to compensate lag and correction of derivative deformation , 2002, Proceedings IEEE Virtual Reality 2002.

[43]  Kosuke Sato,et al.  A wearable mixed reality with an on-board projector , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[44]  Ronald Azuma,et al.  Improving static and dynamic registration in an optical see-through HMD , 1994, SIGGRAPH.

[45]  Ji Zhou,et al.  Evaluation of calibration procedures for optical see-through head-mounted displays , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[46]  Peter F. Sturm,et al.  Algorithms for plane-based pose estimation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[47]  Ronald Azuma,et al.  Hybrid inertial and vision tracking for augmented reality registration , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[48]  Eric Foxlin,et al.  Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[49]  Tom Drummond,et al.  Dynamic measurement clustering to aid real time tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[50]  J. L. Roux.  An Introduction to the Kalman Filter , 2003 .

[51]  Huei-Yung Lin,et al.  Vehicle Speed Detection and Identification from a Single Motion Blurred Image , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[52]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[53]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[54]  Andrew Calway,et al.  Real-Time Camera Tracking Using a Particle Filter , 2005, BMVC.

[55]  Tomás Cipra,et al.  Robust Kalman filter and its application in time series analysis , 1991, Kybernetika.

[56]  Bernd Kleinjohann,et al.  Mobile AR4ALL , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[57]  Nicolas Franceschini,et al.  Visual Guidance Of A Mobile Robot Equipped With A Network Of Self-Motion Sensors , 1990, Other Conferences.

[58]  David E. Breen,et al.  Interactive Occlusion and Automatic Object Placement for Augmented Reality , 1996, Comput. Graph. Forum.

[59]  Khoi Nguyen,et al.  Computer-vision-based registration techniques for augmented reality , 1996, Other Conferences.

[60]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[61]  Harry M. Assenheim,et al.  Large Screen Projection, Avionic, and Helmet-Mounted Displays , 1991 .

[62]  Fan Xiao,et al.  What is the best fiducial? , 2002, The First IEEE International Workshop Agumented Reality Toolkit,.

[63]  Donald B. Gennery,et al.  Visual tracking of known three-dimensional objects , 1992, International Journal of Computer Vision.

[64]  Oliver Bimber,et al.  Occlusion shadows: using projected light to generate realistic occlusion effects for view-dependent optical see-through displays , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[65]  Oliver Bimber,et al.  Video see-through AR on consumer cell-phones , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[66]  David G. Lowe,et al.  Scene modelling, recognition and tracking with invariant image features , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[67]  Thomas P. Caudell,et al.  Calibration of head-mounted displays for augmented reality applications , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[68]  Andrew W. Fitzgibbon,et al.  Reliable Fiducial Detection in Natural Scenes , 2004, ECCV.

[69]  Paul Smith,et al.  Edge-based motion segmentation , 2002 .

[70]  Axel Pinz,et al.  The integration of optical and magnetic tracking for multi-user augmented reality , 1999, Comput. Graph..

[71]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[72]  Nassir Navab,et al.  Optical see-through HMD calibration: a stereo method validated with a video see-through system , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).