On-line, interactive view synthesis and augmentation

Abstract Our on-line, interactive system for enhanced tele-teaching starts from a network of fixed cameras, placed around an instructor. A ‘virtual’ camera can move freely between these ‘real’ cameras. The creation of a virtual view requires on-line foreground/background segmentation and 3D reconstructions of the foreground. An off-line, user-interactive method is used for the initial 3D reconstruction of the background. An on-line update of the background model is done automatically. This on-line analysis also allows the system to place the instructor in a ‘virtual or mixed environment’. The surroundings can be replaced by a more suitable background, like an outdoor scene or large-sized presentation slides. Given a desktop and a set of consumer grade cameras, a lecture can be given from an arbitrary place, e.g. an office. Virtual objects, in particular ‘virtual post-its’ or labels, can be used to augment the scene. They can be on-line selected slide cut-outs or predefined elements. The former are based on simple gestures with a laser pointer.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Hirokazu Kato,et al.  3-D live: real time interaction for mixed reality , 2002, CSCW '02.

[3]  Hirokazu Kato,et al.  Real World Teleconferencing , 2002, IEEE Computer Graphics and Applications.

[4]  Carlo Tomasi,et al.  A Pixel Dissimilarity Measure That Is Insensitive to Image Sampling , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Andrew Blake,et al.  Gaze manipulation for one-to-one teleconferencing , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[7]  Hirokazu Kato,et al.  Collaborative augmented reality , 2002, CACM.

[8]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[9]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Luc Van Gool,et al.  Hierarchical coarse to fine depth estimation for realistic view interpolation , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[11]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[12]  Ronald Azuma,et al.  Evaluating label placement for augmented reality view management , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[13]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[15]  Til Aach,et al.  Illumination-Invariant Change Detection Using a Statistical Colinearity Criterion , 2001, DAGM-Symposium.

[16]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[17]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[18]  Zhuowen Tu,et al.  Range image segmentation by an effective jump-diffusion method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Xing Chen,et al.  Lumipoint: multi-user laser-based interaction on large tiled displays , 2002 .

[20]  Václav Hlavác,et al.  Rendering real-world objects using view interpolation , 1995, Proceedings of IEEE International Conference on Computer Vision.