Remote walkthrough over mobile networks using 3-D image warping and streaming

A mobile walkthrough environment based on 3-D superview warping and view compensation streaming is presented. In this environment, 3-D views are rendered on the remote clients using 3-D oversized reference views and compensated on a view-on-demand basis by the graphically rendered views from the server. The view compensations (hole residuals and the depth map) are coded using a layered joint source-channel coding with unequal error corrections to protect the significant packets on noisy and erasure mobile channels. The new compensated view will then serve as a reference view for the subsequent rendering on the clients. This environment enables numerous mobile virtual applications, such as remote m-gaming, remote m-design, m-learning, m-CAD/CAM, etc., scalable to different network capabilities.

[1]  Daniel Cohen-Or.  Model-based view-extrapolation for interactive VR Web-systems , 1997, Proceedings Computer Graphics International.

[2]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[3]  Jonathan M. Garibaldi,et al.  Real-Time Correlation-Based Stereo Vision with Reduced Border Errors , 2002, International Journal of Computer Vision.

[4]  Paul Bao,et al.  Superview 3D image warping for visibility gap errors , 2003, IEEE Trans. Consumer Electron..

[5]  Philip Dutré,et al.  Interactive boolean operations on surfel-bounded solids , 2003, ACM Trans. Graph..

[6]  Daniel Cohen-Or,et al.  Selective Pixel Transmission for Navigating in Remote Virtual Environments , 1997, Comput. Graph. Forum.

[7]  Matthias Zwicker,et al.  Surfels: surface elements as rendering primitives , 2000, SIGGRAPH.

[8]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[9]  Ulrich Neumann,et al.  Compression of computer graphics images with image-based rendering , 1998, Electronic Imaging.

[10]  Luigi Rizzo,et al.  Effective erasure codes for reliable computer communication protocols , 1997, CCRV.

[11]  George Wolberg,et al.  Digital image warping , 1990 .

[12]  Karel Culik,et al.  Finite automata based compression of bi-level images , 1996, Proceedings of Data Compression Conference - DCC '96.

[13]  Terry A. Welch,et al.  A Technique for High-Performance Data Compression , 1984, Computer.

[14]  William R. Mark,et al.  Post-Rendering 3D Image Warping: Visibility, Reconstruction, and Performance for Depth-Image Warping , 1999 .

[15]  Zixiang Xiong,et al.  Combined Wavelet Video Coding and Error Control for Internet Streaming and Multicast , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[16]  W. C. Jakes,et al.  Microwave Mobile Communications , 1974 .

[17]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[18]  Zixiang Xiong,et al.  On packetization of embedded multimedia bitstreams , 2001, IEEE Trans. Multim..