Augmented Reality Using Uncalibrated Video Sequences

Augmented Reality (AR) aims at merging the real and the virtual in order to enrich a real environment with virtual information. Augmentations range from simple text annotations accompanying real objects to virtual mimics of real-life objects inserted into a real environment. In the latter case the ultimate goal is to make it impossible to differentiate between real and virtual objects. Several problems need to be overcome before realizing this goal. Amongst them are the rigid registration of virtual objects into the real environment, the problem of mutual occlusion of real and virtual objects and the extraction of the illumination distribution of the real environment in order to render the virtual objects with this illumination model. This paper will unfold how we proceeded to implement an Augmented Reality System that registers virtual objects into a totally uncalibrated video sequence of a real environment that may contain some moving parts. The other problems of occlusion and illumination will not be discussed in this paper but are left as future research topics.

[1]  Katsushi Ikeuchi,et al.  Illumination Distribution from Brightness in Shadows: Adaptive Estimation of Illumination Distribution with Unknown Reflectance Properties in Shadow Regions , 1999, ICCV.

[2]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[3]  Kiriakos N. Kutulakos,et al.  Affine object representations for calibration-free augmented reality , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[4]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Matthias M. Wloka,et al.  Resolving occlusion in augmented reality , 1995, I3D '95.

[6]  Yongduek Seo,et al.  Video augmentation by image-based rendering under the perspective camera model , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[7]  Luc Van Gool,et al.  Content-Based Image Retrieval Based on Local Affinely Invariant Regions , 1999, VISUAL.

[8]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[9]  Marie-Odile Berger.  Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  David E. Breen,et al.  Interactive Occlusion and Collision of Real and Virtual Objects in Augmented Reality , 2000 .

[11]  R. Hartley.  Triangulation, Computer Vision and Image Understanding , 1997 .

[12]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[13]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[14]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[15]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[16]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[17]  Gilles Simon,et al.  Mixing synthesis and video images of outdoor environments: application to the bridges of Paris , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[18]  Takeo Kanade,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995, CVRMed.

[19]  Alain Fournier.  Illumination Problems in Computer Augmented Reality , 1994 .

[20]  Jean-Philippe Tarel,et al.  RES: computing the interactions between real and virtual objects in video sequences , 1995 .

[21]  Heinz Huegli,et al.  Augmented reality using range images , 1997, Electronic Imaging.

[22]  William E. Lorensen,et al.  The visualization toolkit (2nd ed.): an object-oriented approach to 3D graphics , 1998 .

[23]  David E. Breen,et al.  Calibration Requirements and Procedures for Augmented Reality , 1997 .

[24]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[25]  R. Hartley.  Cheirality Invariants , 1993 .

[26]  David E. Breen,et al.  Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System , 1995, IEEE Trans. Vis. Comput. Graph..

[27]  Roberto Cipolla,et al.  Computer Vision — ECCV '96 , 1996, Lecture Notes in Computer Science.

[28]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[29]  大野 義夫,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[30]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[31]  M. Pollefeys.  Self-calibration and metric 3d reconstruction from uncalibrated image sequences , 1999 .

[32]  G. Sandini,et al.  Computer Vision — ECCV'92 , 1992, Lecture Notes in Computer Science.

[33]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.