Stable real-time interaction between virtual humans and real scenes

We present an augmented reality system that relies on purely passive techniques to solve the real-time registration problem. It can run on a portable PC and does not require engineering of the environment, for example by adding markers. To achieve this result, we have integrated robust computer vision techniques into a powerful VR framework. The resulting AR system allows us to produce complex rendering and animation of virtual human characters, and to blend them into the real world. The system tracks the 3D camera position by means of a natural features tracker, which, given a rough CAD model, can deal with complex 3D objects. The tracking method can handle both large camera displacements and aspect changes. We will show that our system works in the cluttered environment of a real industrial facility and can, therefore, be used to enhance manufacturing and industrial processes.

[1]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[2]  Yakup Genc,et al.  Marker-less tracking for AR: a learning-based approach , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[3]  Roberto Cipolla,et al.  Real-Time Tracking of Multiple Articulated Structures in Multiple Views , 2000, ECCV.

[4]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[5]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[6]  Jing Xiao,et al.  Robust full-motion recovery of head by dynamic templates and re-registration techniques , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[7]  Suya You,et al.  Natural Feature Tracking for Augmented Reality , 1999, IEEE Trans. Multim..

[8]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[9]  Vincent Lepetit,et al.  Fusing online and offline information for stable 3D tracking in real-time , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Patrick Bouthemy,et al.  Robust real-time visual tracking using a 2D-3D model-based approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Bruce A. Draper,et al.  Adaptive tracking and model registration across distinct aspects , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[14]  Marie-Odile Berger,et al.  Real time registration of known or recovered multi-planar structures: application to AR , 2002, BMVC.