Integrated approach of streaming 3d multimedia contents in real-time for mobile devices

It is popular to watch a 3D video through a 3D display nowadays. However, it is still difficult to enjoy the 3D multimedia contents with a mobile device even if a mobile device with a 3D display is currently introduced into the market. The main technological challenges for watching 3D contents via the mobile devices can be identified as the following: generating and streaming 3D contents. Generating 3D contents requires extra computational resources. Moreover, streaming 3D contents demands additional network bandwidth for receiving and transmitting the 3D data. To overcome these technological challenges, we propose ReMA, a novel 3D video streaming system in this paper. We devised a novel architecture for transmitter, receiver, and a distribution system to efficiently disseminate and generate 3D videos for the mobile devices. We implemented ReMA in a real test-bed and conducted a thorough empirical evaluation study to see the feasibility of streaming 3D contents for the mobile devices. Based on our empirical study, the resulting system presents a great promise in streaming 3D video in real-time to the mobile devices.

[1]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[2]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[3]  A. Murat Tekalp,et al.  Evaluation of Asymmetric Stereo Video Coding and Rate Scaling for Adaptive 3D Video Streaming , 2011, IEEE Transactions on Broadcasting.

[4]  Thomas Stockhammer,et al.  Dynamic adaptive streaming over HTTP --: standards and design principles , 2011, MMSys.

[5]  Heiko Schwarz,et al.  3D High-Efficiency Video Coding for Multi-View Video and Depth Data , 2013, IEEE Transactions on Image Processing.

[6]  Kannan Ramchandran,et al.  PRISM: A new robust video coding architecture based on distributed compression principles , 2002 .

[7]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Hwangnam Kim,et al.  Devising a user collaboration scheme to automatically generate videos , 2015, Multimedia Tools and Applications.

[9]  Ahmet M. Kondoz,et al.  Joint source and channel coding for 3D video with depth image - based rendering , 2008, IEEE Transactions on Consumer Electronics.

[10]  I. Sexton,et al.  Stereoscopic and autostereoscopic display systems , 1999, IEEE Signal Process. Mag..

[11]  Andrew R. Nix,et al.  Raptor Code-Aware Link Adaptation for Spectrally Efficient Unicast Video Streaming over Mobile Broadband Networks , 2015, IEEE Transactions on Mobile Computing.

[12]  Takeshi Oishi,et al.  Image-Based Network Rendering of Large Meshes for Cloud Computing , 2010, International Journal of Computer Vision.

[13]  Srecko Kunic,et al.  3D television , 2011, Proceedings ELMAR-2011.

[14]  Chong Luo,et al.  Multimedia Cloud Computing , 2011, IEEE Signal Processing Magazine.

[15]  Marco Fiorentino,et al.  A multi-directional backlight for a wide-angle, glasses-free three-dimensional display , 2013, Nature.

[16]  NohJunyong,et al.  Video Panorama for 2D to 3D Conversion , 2012 .

[17]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[18]  Eitan Marder-Eppstein,et al.  Project Tango , 2016, SIGGRAPH Real-Time Live!.

[19]  Woonghee Lee,et al.  ReMA: Real-time 3D video streaming system for mobile devices , 2014, 2014 IEEE 33rd International Performance Computing and Communications Conference (IPCCC).

[20]  Yang Lu,et al.  SHARC: A scalable 3D graphics virtual appliance delivery framework in cloud , 2011, J. Netw. Comput. Appl..

[21]  Neil A. Dodgson,et al.  Autostereoscopic 3D displays , 2005, Computer.

[22]  Aljoscha Smolic,et al.  Reliability-based generation and view synthesis in layered depth video , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[23]  M. Halle.  Autostereoscopic displays and computer graphics , 1997, COMG.

[24]  Yuedong Xu,et al.  Analytical QoE Models for Bit-Rate Switching in Dynamic Adaptive Streaming Systems , 2014, IEEE Transactions on Mobile Computing.

[25]  Wojciech Matusik,et al.  Anahita: A System for 3D Video Streaming with Depth Customization , 2014, ACM Multimedia.

[26]  Graham John Woodgate,et al.  A prototype 3D mobile phone equipped with a next-generation autostereoscopic display , 2007, Electronic Imaging.

[27]  Ali C. Begen,et al.  An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP , 2011, MMSys.

[28]  Jeffrey S. McVeigh,et al.  Intermediate view synthesis considering occluded and ambiguously referenced image regions , 1996, Signal Process. Image Commun..

[29]  J. F. Algorri,et al.  An Autostereoscopic Device for Mobile Applications Based on a Liquid Crystal Microlens Array and an OLED Display , 2014, Journal of Display Technology.

[30]  Hwangnam Kim,et al.  Link-Aware Reconfigurable Point-to-Point Video Streaming for Mobile Devices , 2015, TOMM.

[31]  Aljoscha Smolic,et al.  Toward a 3D video format for auto-stereoscopic displays , 2008, Optical Engineering + Applications.

[32]  Chuohao Yeo,et al.  View Synthesis for Robust Distributed Video Compression in Wireless Camera Networks , 2007, 2007 IEEE International Conference on Image Processing.

[33]  Younghui Kim,et al.  Video Panorama for 2D to 3D Conversion , 2012, Comput. Graph. Forum.

[34]  Aljoscha Smolic,et al.  Coding and intermediate view synthesis of multiview video plus depth , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).