Novel view generation from multiple omni-directional videos

Recently, generation of novel views from images acquired by multiple cameras has been investigated in the fields of virtual and mixed reality. Most conventional methods need some assumptions about the scene such as a static scene and limited positions of objects. In this paper, we propose a new method for generating novel view images of a dynamic scene with a wide view, which does not depend on the scene. The images acquired from omni-directional cameras are first divided into static regions and dynamic regions. The novel view images are then generated by applying a morphing technique to static regions and by computing visual hulls for dynamic regions in realtime. In experiments, we show that a prototype system can generate novel view images in real-time from live video streams.

[1]  Hideo Saito,et al.  Immersive evaluation of virtualized soccer match at real stadium model , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[2]  Naokazu Yokoya,et al.  Telepresence by Real-Time View-Dependent Image Generation from Omnidirectional Video Streams , 1998, Comput. Vis. Image Underst..

[3]  Yutaka Nakamura,et al.  Networked telepresence system using web browsers and omni-directional video streams , 2005, IS&T/SPIE Electronic Imaging.

[4]  H. Baker,et al.  Three-Dimensional Modeling , 2019, IJCAI.

[5]  Naokazu Yokoya,et al.  Arbitrary viewpoint rendering from multiple omnidirectional images for interactive walkthroughs , 2002, Object recognition supported by user interaction for service robots.

[6]  Naokazu Yokoya,et al.  Networked video surveillance using multiple omnidirectional cameras , 2003, Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium (Cat. No.03EX694).

[7]  Takeo Kanade,et al.  Appearance-based virtual view generation from multicamera videos captured in the 3-D room , 2003, IEEE Trans. Multim..

[8]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[9]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[10]  I. Kitahara,et al.  Live mixed-reality 3D video in soccer stadium , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..