System design of free viewpoint video communication

We propose a free viewpoint video communication method that allows the user to change his/her viewing point and viewing direction freely. This application will be a next generation visual application in the near future. We propose suitable multiple view video coding method and communication protocol for this communication application, and demonstrate the coding efficiency of the proposed method. We also demonstrate that the developed viewer can generate the view of arbitrary viewpoint and view direction. This viewer uses the Ray-Space interpolation and extrapolation methods and generates a natural view.

[1]  Toshiaki Fujii,et al.  Ray space coding for 3D visual communication , 1996 .

[2]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[3]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[4]  Henning Schulzrinne,et al.  Real Time Streaming Protocol (RTSP) , 1998, RFC.

[5]  Thomas Wiegand,et al.  Draft ITU-T recommendation and final draft international standard of joint video specification , 2003 .

[6]  Keith W. Ross,et al.  A join-the-shortest-queue prefetching protocol for VBR video on demand , 1997, Proceedings 1997 International Conference on Network Protocols.

[7]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[8]  Henning Schulzrinne,et al.  RTP: A Transport Protocol for Real-Time Applications , 1996, RFC.

[9]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[10]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Jens-Rainer Ohm,et al.  Incomplete 3D for Multiview Representation and Synthesis of Video Objects , 1998, ECMAST.

[12]  Hideaki Kimata,et al.  Hierarchical reference picture selection method for temporal scalability beyond H.264 , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[13]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[14]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[15]  Toshiaki Fujii,et al.  Ray-space coding based on arbitrarily shaped DCT , 1998, Electronic Imaging.

[16]  Toshiaki Fujii,et al.  Interpolation of ray-space data by adaptive filtering , 2000, Electronic Imaging.

[17]  Wei-Chao Chen,et al.  Light field mapping: efficient representation and hardware rendering of surface light fields , 2002, SIGGRAPH.

[18]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[19]  Jens-Rainer Ohm.  - 1-Incomplete 3 D Representation of Video Objects for Multiview Applications , 1999 .