Real-time Free-Viewpoint Navigation from Compressed Multi-Video Recordings

We present an interactive player for free-viewpoint video. Free-viewpoint video data typically consist of large amounts of video datasets captured from multiple cameras and additional inter-view and frame information. Neither the size nor the representation of this data is appropriate for real-time rendering. Inspired by a new image interpolation approach based on correspondence fields, we propose a data compression algorithm for streamable multi-view video sequences allowing a real-time decoding and playback for videos of arbitrary length without loading them completely into cache.

[1]  Thomas Wiegand,et al.  3D Video and Free Viewpoint Video - Technologies, Applications and MPEG Standards , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[2]  Carlo Tomasi,et al.  Color edge detection with the compass operator , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[4]  Wojciech Matusik,et al.  3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes , 2004, ACM Trans. Graph..

[5]  Marcus A. Magnor,et al.  Perception-motivated interpolation of image sequences , 2008, APGV.

[6]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[7]  Marcus A. Magnor,et al.  Data compression for light-field rendering , 2000, IEEE Trans. Circuits Syst. Video Technol..

[8]  C. Lawrence Zitnick,et al.  High-quality video view interpolation , 2006, SIGGRAPH Courses.

[9]  Martin Burtscher,et al.  Fast lossless compression of scientific floating-point data , 2006, Data Compression Conference (DCC'06).

[10]  Marcus A. Magnor,et al.  Multi-view coding for image-based rendering using 3-D scene geometry , 2003, IEEE Trans. Circuits Syst. Video Technol..

[11]  Aljoscha Smolic,et al.  Efficient Compression of Multi-View Video Exploiting Inter-View Dependencies Based on H.264/MPEG4-AVC , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[12]  Marcus A. Magnor,et al.  View and Time Interpolation in Image Space , 2008, Comput. Graph. Forum.

[13]  André Kaup,et al.  Wavelet-based multi-view video coding with joint best basis wavelet packets , 2008, 2008 15th IEEE International Conference on Image Processing.

[14]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[15]  Yo-Sung Ho,et al.  Preprocessing of Depth and Color Information for Layered Depth Image Coding , 2004, PCM.

[16]  Lifeng Sun,et al.  A multi-view video coding approach using Layered Depth Image , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[17]  Martin Isenburg,et al.  Fast and Efficient Compression of Floating-Point Data , 2006, IEEE Transactions on Visualization and Computer Graphics.

[18]  Leif Kobbelt,et al.  Interactive Pixel‐Accurate Free Viewpoint Rendering from Images with Silhouette Aware Sampling , 2009, Comput. Graph. Forum.

[19]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[20]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.