Markerless augmented reality with a real-time affine region tracker

We present a system for planar augmented reality based on a new real-time affine region tracker. Instead of tracking fiducial points, we track planar local image patches, and bring these into complete correspondence, so a virtual texture can directly be added to them. Moreover, the local image patches can be extracted in an invariant way, even without any a priori information from previous frames. Hence, it is possible to use them as natural beacons, that can be used to recognize the scene and to identify the individual patches. This results in a powerful system that can work without artificial markers or fiducial points and with a minimal amount of user interference.

[1]  Ramesh C. Jain,et al.  Using Dynamic Programming for Solving Variational Problems in Vision , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Naonori Ueda,et al.  Tracking Moving Contours Using Energy-Minimizing Elastic Contour Models , 1992, ECCV.

[3]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  .. McGlone,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995 .

[5]  Rachid Deriche,et al.  Region tracking through image sequences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[7]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[8]  Li-Te Cheng,et al.  Dealing with speed and robustness issues for video-based registration on a wearable computing platform , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[9]  Stanley T. Birchfield,et al.  Elliptical head tracking using intensity gradients and color histograms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[10]  Hideyuki Tamura,et al.  AR2 Hockey , 1998, SIGGRAPH '98.

[11]  Hideyuki Tamura,et al.  AR/sup 2/Hockey: a case study of collaborative augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[12]  W. A. Ho,et al.  Fusion of data from head-mounted and fixed sensors , 1998 .

[13]  Venkataraman Sundareswaran,et al.  Visual servoing-based augmented reality , 1999 .

[14]  Klaus Dorfmüller,et al.  Robust tracking for augmented reality using retroreflective markers , 1999, Comput. Graph..

[15]  Luc Van Gool,et al.  Content-Based Image Retrieval Based on Local Affinely Invariant Regions , 1999, VISUAL.

[16]  Suya You,et al.  Natural Feature Tracking for Augmented Reality , 1999, IEEE Trans. Multim..

[17]  Luc Van Gool,et al.  Recognizing color patterns irrespective of viewpoint and illumination , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[18]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Jun Rekimoto,et al.  CyberCode: designing augmented reality environments with visual tags , 2000, DARE '00.

[20]  Luc Van Gool,et al.  Wide Baseline Stereo Matching based on Local, Affinely Invariant Regions , 2000, BMVC.

[21]  Wolfgang Broll,et al.  The virtual round table - a collaborative augmented multi-user environment , 2000, CVE '00.

[22]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[23]  Katsuhiko Sakaue,et al.  Real-time camera parameter estimation from images for a mixed reality system , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[24]  Axel Pinz,et al.  A new optical tracking system for virtual and augmented reality applications , 2001, IMTC 2001. Proceedings of the 18th IEEE Instrumentation and Measurement Technology Conference. Rediscovering Measurement in the Age of Informatics (Cat. No.01CH 37188).

[25]  Desney S. Tan,et al.  The best of two worlds: merging virtual and real for face to face collaboration , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[26]  [6] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. Int. , .