Toward Practical Volumetric Video Streaming on Commodity Smartphones

Volumetric videos offer six degree-of-freedom (DoF) as well as 3D rendering, making them highly immersive, interactive, and expressive. In this paper, we design Nebula, a practical and resource-efficient volumetric video streaming system for commodity mobile devices. Our design leverages edge computing to reduce the computation burden on mobile clients. We also introduce various optimizations to lower the perceived "motion-to-photon" delay, to dynamically adapt to the fluctuating network bandwidth, and to reduce the system's resource consumption while maintaining a high QoE.

[1]  Leif Kobbelt,et al.  A survey of point-based techniques in computer graphics , 2004, Comput. Graph..

[2]  Reinhard Klein,et al.  Eurographics Symposium on Point-based Graphics (2006) Octree-based Point-cloud Compression , 2022 .

[3]  Meenakshisundaram Gopi,et al.  A Generic Scheme for Progressive Point Cloud Coding , 2008, IEEE Transactions on Visualization and Computer Graphics.

[4]  Nico Blodow,et al.  Real-time compression of point cloud streams , 2012, 2012 IEEE International Conference on Robotics and Automation.

[5]  Minhua Zhou,et al.  An Overview of Tiles in HEVC , 2013, IEEE Journal of Selected Topics in Signal Processing.

[6]  Anh Nguyen,et al.  3D point cloud segmentation: A survey , 2013, 2013 6th IEEE Conference on Robotics, Automation and Mechatronics (RAM).

[7]  Marek Kowalski,et al.  Livescan3D: A Fast and Inexpensive 3D Data Acquisition System for Multiple Kinect v2 Sensors , 2015, 2015 International Conference on 3D Vision.

[8]  Céline Hudelot,et al.  3D Mesh Compression , 2015, ACM Comput. Surv..

[9]  Reinhard Klein,et al.  Real-time point cloud compression , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Hermann Hellwagner,et al.  A scalable video coding dataset and toolchain for dynamic adaptive streaming over HTTP , 2015, MMSys.

[11]  Bruno Sinopoli,et al.  A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP , 2015, Comput. Commun. Rev..

[12]  Mohamed Hefeeda,et al.  Adaptive streaming of interactive free viewpoint videos to heterogeneous clients , 2016, MMSys.

[13]  David Chu,et al.  FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization , 2016, MobiSys.

[14]  David Chu,et al.  FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization , 2017, GETMBL.

[15]  Y. Charlie Hu,et al.  Furion: Engineering High-Quality Immersive Virtual Reality on Today's Mobile Devices , 2017, IEEE Transactions on Mobile Computing.

[16]  Rufael Mekuria,et al.  Design, Implementation, and Evaluation of a Point Cloud Codec for Tele-Immersive Video , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  C. Bambi,et al.  Fundamental Concepts , 2018, Tutorial Guide to X-ray and Gamma-ray Astronomy.

[18]  Flare , 2018, Proceedings of the 24th Annual International Conference on Mobile Computing and Networking.

[19]  Ramesh Govindan,et al.  Augmented Vehicular Reality: Enabling Extended Vision for Future Automobiles , 2019, GETMBL.

[20]  AVR , 2018, Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services.

[21]  Feng Li,et al.  Rubiks: Practical 360-Degree Streaming for Smartphones , 2018, MobiSys.

[22]  Feng Qian,et al.  Flare: Practical Viewport-Adaptive 360-Degree Video Streaming for Mobile Devices , 2018, MobiCom.

[23]  Touradj Ebrahimi,et al.  Point Cloud Quality Assessment Metric Based on Angular Similarity , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[24]  Ramesh Govindan,et al.  AVR: Augmented Vehicular Reality , 2018, MobiSys.

[25]  Emil Dumic,et al.  Subjective evaluation and objective measures for point clouds — State of the art , 2018, 2018 First International Colloquium on Smart Grid Metrology (SmaGriMet).

[26]  Jenq-Neng Hwang,et al.  Volumetric Media Streaming for Augmented Reality , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[27]  Mohammad Hosseini,et al.  Dynamic Adaptive Point Cloud Streaming , 2018, PV@MMSys.

[28]  Rubiks , 2018, Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services.