Tracking based structure and motion recovery for augmented video productions

Augmented Reality (AR) can hardly be called uncharted territory. Much research in this area revealed solutions to the three most prominent challenges of AR: accurate camera state retrieval, resolving occlusions between real and virtual objects and extraction of environment illumination distribution. Solving these three challenges improves the illusion of virtual entities belonging to our reality. This paper demonstrates an elaborated framework that recovers accurate camera states from a video sequence based on feature tracking. Without prior calibration knowledge, it is able to create AR Video products with negligible/invisible jitter or drift of virtual entities starting from general input video sequences. Together with the referenced papers, this work describes a readily implementable and robust AR-System.

[1]  Marie-Odile Berger.  Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[3]  Alex Pentland,et al.  Recursive Estimation of Motion, Structure, and Focal Length , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[5]  Pietro Perona,et al.  Visual navigation using a single camera , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Katsushi Ikeuchi,et al.  Illumination Distribution from Brightness in Shadows: Adaptive Estimation of Illumination Distribution with Unknown Reflectance Properties in Shadow Regions , 1999, ICCV.

[7]  S. Bougnoux,et al.  From projective to Euclidean space under any practical situation, a criticism of self-calibration , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[8]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[9]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.

[10]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Luc Van Gool,et al.  Content-Based Image Retrieval Based on Local Affinely Invariant Regions , 1999, VISUAL.

[12]  David E. Breen,et al.  Interactive Occlusion and Collision of Real and Virtual Objects in Augmented Reality , 2000 .

[13]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  David E. Breen,et al.  Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System , 1995, IEEE Trans. Vis. Comput. Graph..

[15]  R. Chellappa,et al.  Recursive 3-D motion estimation from a monocular image sequence , 1990 .

[16]  David E. Breen,et al.  Calibration Requirements and Procedures for Augmented Reality , 1997 .

[17]  Heinz Huegli,et al.  Augmented reality using range images , 1997, Electronic Imaging.

[18]  M. Brooks.  An analytical approach to determining the ego-motion of a camera having free intrinsic parameters , 2002 .

[19]  Reinhard Koch,et al.  Hand-held acquisition of 3D models with a video camera , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[20]  Luc Van Gool,et al.  3D Structure from Images — SMILE 2000 , 2001, Lecture Notes in Computer Science.

[21]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[22]  William E. Lorensen,et al.  The visualization toolkit (2nd ed.): an object-oriented approach to 3D graphics , 1998 .

[23]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[24]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Maarten Vergauwen,et al.  Augmented Reality Using Uncalibrated Video Sequences , 2000, SMILE.

[26]  Xinhua Zhuang,et al.  Pose estimation from corresponding point data , 1989, IEEE Trans. Syst. Man Cybern..

[27]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[28]  .. McGlone,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995 .

[29]  Kiriakos N. Kutulakos,et al.  Affine object representations for calibration-free augmented reality , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[30]  Matthias M. Wloka,et al.  Resolving occlusion in augmented reality , 1995, I3D '95.

[31]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[32]  Olivier D. Faugeras,et al.  The First Order Expansion of Motion Equations in the Uncalibrated Case , 1996, Comput. Vis. Image Underst..