POI360: Panoramic Mobile Video Telephony over LTE Cellular Networks

Panoramic or 360° video streaming has been supported by a wide range of content providers and mobile devices. Yet existing work primarily focused on streaming on-demand 360° videos stored on servers. In this paper, we examine a more challenging problem: Can we stream real-time interactive 360° videos across existing LTE cellular networks, so as to trigger new applications such as ubiquitous 360° video chat and panoramic outdoor experience sharing? To explore the feasibility and challenges underlying this vision, we design POI360, a portable interactive 360° video telephony system that jointly investigates both panoramic video compression and responsive video stream rate control. For the challenge that the legacy spatial compression algorithms for 360° video suffer from severe quality fluctuations as the user changes her region-of-interest (ROI), we design an adaptive compression scheme, which dynamically adjusts the compression strategy to stabilize the video quality within ROI under various user input and network condition. In addition, to meet the responsiveness requirement of panoramic video telephony, we leverage the diagnostic statistics on commodity phones to promptly detect cellular link congestion, hence significantly boosting the rate control responsiveness. Extensive field tests for our real-time POI360 prototype validate its effectiveness in enabling panoramic video telephony over the highly dynamic cellular networks.

[1]  Xin Liu,et al.  Motion-Prediction-Based Multicast for 360-Degree Video Transmissions , 2017, 2017 14th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON).

[2]  Qiang Xu,et al.  PROTEUS: network performance forecast for real-time, interactive mobile applications , 2013, MobiSys '13.

[3]  Wei Tsang Ooi,et al.  Towards understanding user tolerance to network latency in zoomable video streaming , 2011, MM '11.

[4]  Tao Wang,et al.  Mobileinsight: extracting and analyzing cellular network information on smartphones , 2016, MobiCom.

[5]  Jacob Chakareski.  Adaptive multiview video streaming: challenges and opportunities , 2013, IEEE Communications Magazine.

[6]  B. S. Manjunath,et al.  Region of interest extraction and virtual camera control based on panoramic video capturing , 2005, IEEE Transactions on Multimedia.

[7]  Te-Yuan Huang,et al.  A buffer-based approach to rate adaptation: evidence from a large video streaming service , 2015, SIGCOMM 2015.

[8]  Bernd Girod,et al.  Mobile interactive region-of-interest video streaming with crowd-driven prefetching , 2011, IMMPD '11.

[9]  Charles T. Loop,et al.  Holoportation: Virtual 3D Teleportation in Real-time , 2016, UIST.

[10]  Michael Riegler,et al.  Tiling in Interactive Panoramic Video: Approaches and Evaluation , 2016, IEEE Transactions on Multimedia.

[11]  Yang Xu,et al.  “Can you SEE me now?” A measurement study of mobile video calls , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[12]  Shreesha Srinath,et al.  Design and Implementation of an “Approximate” Communication System for Wireless Media Applications , 2010, IEEE/ACM Transactions on Networking.

[13]  Mingyu Chen,et al.  Improving QoE for Skype video call in Mobile Broadband Network , 2012, 2012 IEEE Global Communications Conference (GLOBECOM).

[14]  Swarun Kumar,et al.  piStream: Physical Layer Informed Adaptive Video Streaming over LTE , 2015, MobiCom.

[15]  Mikhal Shemer,et al.  Handling packet loss in WebRTC , 2013, 2013 IEEE International Conference on Image Processing.

[16]  Hari Balakrishnan,et al.  Stochastic Forecasts Achieve High Throughput and Low Delay over Cellular Networks , 2013, NSDI.

[17]  Gregory J. Zelinsky,et al.  Design and evaluation of a foveated video streaming service for commodity client devices , 2016, MMSys.

[18]  Rong Pan,et al.  NADA: A Unified Congestion Control Scheme for Low-Latency Interactive Video , 2013, 2013 20th International Packet Video Workshop.

[19]  Vyas Sekar,et al.  Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with FESTIVE , 2012, CoNEXT '12.

[20]  Wei Tsang Ooi,et al.  Adaptive encoding of zoomable video streams based on user access pattern , 2011, MMSys.

[21]  Dan Siemon.  Queueing in the Linux network stack , 2013 .

[22]  Wei Tsang Ooi,et al.  Supporting zoomable video streams with dynamic region-of-interest cropping , 2010, MMSys '10.

[23]  Luca De Cicco,et al.  Skype Video congestion control: An experimental investigation , 2011, Comput. Networks.

[24]  Feng Qian,et al.  An in-depth study of LTE: effect of network protocol and application behavior on performance , 2013, SIGCOMM.

[25]  Jean-François Macq,et al.  Interactive omnidirectional video delivery: A bandwidth-effective approach , 2012, Bell Labs Technical Journal.

[26]  Ronald Azuma,et al.  Predictive tracking for augmented reality , 1995 .

[27]  David Chu,et al.  Poster: Mobile Virtual Reality for Head-mounted Displays With Interactive Streaming Video and Likelihood-based Foveation , 2016, MobiSys '16 Companion.

[28]  Steven M. LaValle,et al.  Head tracking for the Oculus Rift , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Feng Qian,et al.  Optimizing 360 video delivery over cellular networks , 2016, ATC@MobiCom.

[30]  Iraj Sodagar,et al.  The MPEG-DASH Standard for Multimedia Streaming Over the Internet , 2011, IEEE MultiMedia.

[31]  Bernd Girod,et al.  Real-time video streaming with interactive region-of-interest , 2010, 2010 IEEE International Conference on Image Processing.

[32]  Feng Qian,et al.  Understanding On-device Bufferbloat for Cellular Upload , 2016, Internet Measurement Conference.

[33]  Luca De Cicco,et al.  A Google Congestion Control Algorithm for Real-Time Communication , 2012 .