3D video communications: Challenges and opportunities

This paper surveys major techniques in 3D communications area, which covers the whole pipeline of the 3D video communication framework, including 3D content creation, data representation, compression, delivery, decompression, post-processing, and 3D scene rendering stages. Both the current state-of-the-art, stereo 3D, and future trend, free-viewpoint 3D, are demonstrated in details. On the other hand, the paper highlights a few features in the emerging 4G wireless systems that are critical for 3D communications system design. At the end, the topics with potential but challenges, for example 3D over 4G networks, distributed 3D video coding, 3D multi-user communication, scalability and universal 3D access, are discussed and pointed out to audiences for further investigation. Copyright © 2010 John Wiley & Sons, Ltd.

[1]  Jie Hui,et al.  Quality of service in WiMAX and LTE networks [Topics in Wireless Communications] , 2010, IEEE Communications Magazine.

[2]  Aljoscha Smolic,et al.  The effects of multiview depth video compression on multiview rendering , 2009, Signal Process. Image Commun..

[3]  M. Halle,et al.  3-D Displays and Signal Processing , 2007, IEEE Signal Processing Magazine.

[4]  Levent Onural,et al.  Three-Dimensional Television , 2008 .

[5]  N. Atzpadin,et al.  Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability , 2007, Signal Process. Image Commun..

[6]  Jiang Li,et al.  Interactive Multiview Video Delivery Based on IP Multicast , 2007, Adv. Multim..

[7]  T. Sikora,et al.  Feasibility of Multi-View Video Streaming Over P2P Networks , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[8]  Ismo Rakkolainen,et al.  A Survey of 3DTV Displays: Techniques and Technologies , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Thomas Wiegand,et al.  Compressing Time-Varying Visual Content , 2007, IEEE Signal Processing Magazine.

[10]  Thomas Wiegand,et al.  Mixed resolution coding of stereoscopic video for Mobile devices , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[11]  A. Murat Tekalp,et al.  Transport Methods in 3DTV—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Aljoscha Smolic,et al.  Interactive 3-D Video Representation and Coding Technologies , 2005, Proceedings of the IEEE.

[13]  Aljoscha Smolic,et al.  Coding Algorithms for 3DTV—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  André Vincent,et al.  Stereo image quality: effects of mixed spatio-temporal resolution , 2000, IEEE Trans. Circuits Syst. Video Technol..

[15]  Lyndon Hill,et al.  3-D Liquid Crystal Displays and Their Applications , 2006, Proceedings of the IEEE.

[16]  Mourad Ouaret,et al.  Iterative Multiview Side Information for Enhanced Reconstruction in Distributed Video Coding , 2009, EURASIP J. Image Video Process..

[17]  Sehoon Yea,et al.  View synthesis prediction for multiview video coding , 2009, Signal Process. Image Commun..

[18]  Wen Gao,et al.  Wyner–Ziv-Based Multiview Video Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Namho Hur,et al.  Depth-image-based rendering for 3DTV service over T-DMB , 2009, Signal Process. Image Commun..

[20]  Carl James Debono,et al.  Minimizing Uplink Data in Wireless Free-Viewpoint Video Transmission Applications , 2009, 2009 Third International Conference on Next Generation Mobile Applications, Services and Technologies.

[21]  Gozde Bozdagi Akar,et al.  Rate-Distortion Optimization for Stereoscopic Video Streaming with Unequal Error Protection , 2009, EURASIP J. Adv. Signal Process..

[22]  Guan-Ming Su,et al.  A Scalable Multiuser Framework for Video Over OFDM Networks: Fairness and Efficiency , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Jean-Yves Guillemaut,et al.  Objective Quality Assessment in Free-Viewpoint Video Production , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[24]  Oliver Schreer,et al.  3D Videocommunication: Algorithms, concepts and real-time systems in human centred communication , 2005 .

[25]  Karen O. Egiazarian,et al.  Classification and simulation of stereoscopic artifacts in mobile 3DTV content , 2009, Electronic Imaging.

[26]  Peter Schelkens,et al.  Scalable Joint Source and Channel Coding of Meshes , 2008, IEEE Transactions on Multimedia.

[27]  Y. Wang,et al.  Optimization and comparision of coding algorithms for mobile 3DTV , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[28]  Ahmet M. Kondoz,et al.  Scalable multiple description video coding for stereoscopic 3D , 2008, IEEE Transactions on Consumer Electronics.

[29]  Mei Yu,et al.  A Transmission and Interaction Oriented Free-Viewpoint Video System , 2007 .

[30]  Wa James Tam,et al.  Stereoscopic image coding: Effect of disparate image-quality in left- and right-eye views , 1998, Signal Process. Image Commun..

[31]  C. Guillemot,et al.  Distributed Monoview and Multiview Video Coding , 2007, IEEE Signal Processing Magazine.

[32]  C. Fehn,et al.  Interactive 3-DTV-Concepts and Key Technologies , 2006 .

[33]  Chuohao Yeo,et al.  Robust Distributed Multiview Video Compression for Wireless Camera Networks , 2010, IEEE Transactions on Image Processing.

[34]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[35]  A. Murat Tekalp,et al.  Client-Driven Selective Streaming of Multiview Video for Interactive 3DTV , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Yücel Altunbasak,et al.  An unequal error protection method for progressively transmitted 3D models , 2005, IEEE Transactions on Multimedia.

[37]  Chaminda T. E. R. Hewage,et al.  Robust 3D video communications , 2009 .

[38]  Guan-Ming Su,et al.  Multiuser cross-layer resource allocation for video transmission over wireless networks , 2006, IEEE Network.

[39]  Sassan Ahmadi,et al.  An overview of next-generation mobile WiMAX technology , 2009, IEEE Communications Magazine.

[40]  Jeffrey G. Andrews,et al.  Femtocell networks: a survey , 2008, IEEE Communications Magazine.

[41]  Fernando Pereira,et al.  MPEG-A: multimedia application formats , 2005, IEEE MultiMedia.

[42]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[43]  Raj Jain,et al.  Modeling and resource allocation for mobile video over WiMAX broadband wireless networks , 2010, IEEE Journal on Selected Areas in Communications.

[44]  Dominik Strohmeier,et al.  Sharp, bright, three-dimensional: open profiling of quality for mobile 3DTV coding methods , 2010, Electronic Imaging.

[45]  Aljoscha Smolic,et al.  View Synthesis for Advanced 3D Video Systems , 2008, EURASIP J. Image Video Process..

[46]  Aljoscha Smolic,et al.  Scene Representation Technologies for 3DTV—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[47]  Wojciech Matusik,et al.  3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes , 2004, ACM Trans. Graph..

[48]  Douglas N. Knisely,et al.  Standardization of femtocells in 3GPP2 , 2009, IEEE Communications Magazine.

[49]  Guan-Ming Su,et al.  Auction-Based Resource Allocation for Cooperative Video Transmission Protocols over Wireless Networks , 2009, EURASIP J. Adv. Signal Process..

[50]  Jin Wang,et al.  Depth Image-Based Temporal Error Concealment for 3-D Video Transmission , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[51]  Wijnand A. IJsselsteijn,et al.  A survey of perceptual evaluations and requirements of three-dimensional TV , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[52]  Song Ci,et al.  Quality-driven cross-layer optimized video delivery over LTE , 2010, IEEE Communications Magazine.

[53]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[54]  Qionghai Dai,et al.  A Flexible Client-Driven 3DTV System for Real-Time Acquisition, Transmission, and Display of Dynamic Scenes , 2009, EURASIP J. Adv. Signal Process..

[55]  Stefan Parkvall,et al.  LTE: the evolution of mobile broadband , 2009, IEEE Communications Magazine.

[56]  John Stenfelt,et al.  Policy and charging control in the evolved packet system , 2009, IEEE Communications Magazine.

[57]  Euee S. Jang,et al.  An introduction to the MPEG-4 animation framework eXtension , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[58]  Ronny Yongho Kim,et al.  WiMAX femtocell: requirements, challenges, and solutions , 2009, IEEE Communications Magazine.

[59]  T. Freeman,et al.  Motion perception during sinusoidal smooth pursuit eye movements: signal latencies and non-linearities. , 2008, Journal of vision.

[60]  In Kyu Park,et al.  Depth image-based representation and compression for static and animated 3-D objects , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[61]  A.M. Tekalp,et al.  3DTV over IP , 2007, IEEE Signal Processing Magazine.

[62]  Ahmet M. Kondoz,et al.  Quality Evaluation of Color Plus Depth Map-Based Stereoscopic Video , 2009, IEEE Journal of Selected Topics in Signal Processing.

[63]  Ahmet M. Kondoz,et al.  Joint source and channel coding for 3D video with depth image - based rendering , 2008, IEEE Transactions on Consumer Electronics.

[64]  B. Girod,et al.  Multiview Video Compression , 2007, IEEE Signal Processing Magazine.

[65]  Bruno Clerckx,et al.  MIMO techniques in WiMAX and LTE: a feature overview , 2010, IEEE Communications Magazine.

[66]  Haohong Wang,et al.  Cross-layer optimization for video summary transmission over wireless networks , 2007, IEEE Journal on Selected Areas in Communications.

[67]  K. J. Ray Liu,et al.  Incentive Cooperation Strategies for Peer-to-Peer Live Multimedia Streaming Social Networks , 2009, IEEE Transactions on Multimedia.

[68]  K. J. Ray Liu,et al.  Cooperation Stimulation Strategies for Peer-to-Peer Wireless Live Video-Sharing Social Networks , 2010, IEEE Transactions on Image Processing.

[69]  Aljoscha Smolic,et al.  Reliability-based generation and view synthesis in layered depth video , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[70]  Ahmet M. Kondoz,et al.  Quality analysis for 3D video using 2D video quality models , 2008, IEEE Transactions on Consumer Electronics.