A stereo vision-based augmented reality system with marker and natural feature tracking

Proposes a method to extend the registration range of a vision-based augmented reality system. We propose to use natural feature points contained in images captured by a pair of stereo cameras in conjunction with pre-defined fixed fiducial markers. The system also incorporates an inertial sensor to achieve a robust registration method which can handle user's fast head rotation and movement. The system first uses pre-defined fiducial markers to estimate a projection matrix between real and virtual coordinate systems. At the same time, the system picks up and tracks a set of natural feature points from the initial image. As a user moves around in an AR environment, the initial markers fall out from the camera frame and natural features are then used to recover the projection matrix. Experiments evaluating the feasibility, of the method are carried out and show the potential benefits of the method.

[1]  Hans P. Moravec.  Visual Mapping by a Robot Rover , 1979, IJCAI.

[2]  Hideyuki Tamura,et al.  AR/sup 2/Hockey: a case study of collaborative augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[3]  Naokazu Yokoya,et al.  A stereo vision-based augmented reality system with an inertial sensor , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[4]  .. McGlone,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995 .

[5]  Hideyuki Tamura,et al.  AR2 Hockey , 1998, SIGGRAPH '98.

[6]  Hirokazu Kato,et al.  Collaborative augmented reality , 2002, CACM.

[7]  Tsuneo Yoshikawa,et al.  Accurate image overlay on video see-through HMDs using vision and accelerometers , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[8]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[9]  Mary C. Whitton,et al.  Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies , 1996, SIGGRAPH.

[10]  Jun Rekimoto,et al.  Matrix: a realtime object identification and registration method for augmented reality , 1998, Proceedings. 3rd Asia Pacific Computer Human Interaction (Cat. No.98EX110).

[11]  Naokazu Yokoya,et al.  A stereoscopic video see-through augmented reality system based on real-time vision-based registration , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[12]  Naokazu Yokoya,et al.  An augmented reality system using a real-time vision based registration , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[13]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.