Scene modelling, recognition and tracking with invariant image features

We present a complete system architecture for fully automated markerless augmented reality (AR). The system constructs a sparse metric model of the real-world environment, provides interactive means for specifying the pose of a virtual object, and performs model-based camera tracking with visually pleasing augmentation results. Our approach does not require camera pre-calibration, prior knowledge of scene geometry, manual initialization of the tracker or placement of special markers. Robust tracking in the presence of occlusions and scene changes is achieved by using highly distinctive natural features to establish image correspondences.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[4]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[5]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[6]  S. B. Kang,et al.  Recovering 3 D Shape and Motion from Image Streams using Non-Linear Least Squares , 1993 .

[7]  Richard Szeliski,et al.  Recovering 3D shape and motion from image streams using nonlinear least squares , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  David E. Breen,et al.  Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System , 1995, IEEE Trans. Vis. Comput. Graph..

[9]  Khoi Nguyen,et al.  Computer-vision-based registration techniques for augmented reality , 1996, Other Conferences.

[10]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[12]  Harpreet Sawhney,et al.  Multi-view 3D estimation and applications to match move , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[13]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[14]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  Yongduek Seo,et al.  Calibration-Free Augmented Reality in Perspective , 2000, IEEE Trans. Vis. Comput. Graph..

[16]  Nassir Navab,et al.  Registration of technical drawings and calibrated images for industrial augmented reality , 2000, Proceedings Fifth IEEE Workshop on Applications of Computer Vision.

[17]  Maarten Vergauwen,et al.  Augmented Reality Using Uncalibrated Video Sequences , 2000, SMILE.

[18]  Luc Van Gool,et al.  Markerless augmented reality with a real-time affine region tracker , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[19]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[20]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[21]  Yakup Genc,et al.  Marker-less tracking for AR: a learning-based approach , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[22]  Andrew Zisserman,et al.  Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps?" , 2002, ECCV.

[23]  Nassir Navab,et al.  Registration of technical drawings and calibrated images for industrial augmented reality , 2002, Machine Vision and Applications.

[24]  Adrian David Cheok,et al.  Online 6 DOF augmented reality registration from natural features , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[25]  Andrew Calway,et al.  Robust Estimation of 3-D Camera Motion for Uncalibrated Augmented Reality , 2002 .

[26]  Vincent Lepetit,et al.  Fully automated and stable registration for augmented reality applications , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[27]  Tom Drummond,et al.  Robust visual tracking for non-instrumental augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[28]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .