Furion: Engineering High-Quality Immersive Virtual Reality on Today's Mobile Devices

In this paper, we perform a systematic design study of the "elephant in the room" facing the VR industry -- is it feasible to enable high-quality VR apps on untethered mobile devices such as smartphones? Our quantitative, performance-driven design study makes two contributions. First, we show that the QoE achievable for high-quality VR applications on today's mobile hardware and wireless networks via local rendering or offloading is about 10X away from the acceptable QoE, yet waiting for future mobile hardware or next-generation wireless networks (e.g. 5G) is unlikely to help, because of power limitation and the higher CPU utilization needed for processing packets under higher data rate. Second, we present Furion, a VR framework that enables high-quality, immersive mobile VR on today's mobile devices and wireless networks. Furion exploits a key insight about the VR workload that foreground interactions and background environment have contrasting predictability and rendering workload, and employs a split renderer architecture running on both the phone and the server. Supplemented with video compression, use of panoramic frames, and parallel decoding on multiple cores on the phone, we demonstrate Furion can support high-quality VR apps on today's smartphones over WiFi, with under 14ms latency and 60 FPS (the phone display refresh rate).

[1]  Desney S. Tan,et al.  Foveated 3D graphics , 2012, ACM Trans. Graph..

[2]  Ryan Shea,et al.  On GPU pass-through performance for cloud gaming: Experiments and analysis , 2013, 2013 12th Annual Workshop on Network and Systems Support for Games (NetGames).

[3]  Omid Salehi-Abari,et al.  Cutting the Cord in Virtual Reality , 2016, HotNets.

[4]  Kurt Keutzer,et al.  Efficient Parallelization of H.264 Decoding with Macro Block Level Scheduling , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[5]  Ryan Shea,et al.  Rhizome: utilizing the public cloud to provide 3D gaming infrastructure , 2015, MMSys.

[6]  Byung-Gon Chun,et al.  CloneCloud: elastic execution between mobile device and cloud , 2011, EuroSys '11.

[7]  Feng Qian,et al.  Optimizing 360 video delivery over cellular networks , 2016, ATC@MobiCom.

[8]  Alec Wolman,et al.  MAUI: making smartphones last longer with code offload , 2010, MobiSys '10.

[9]  Michael Riegler,et al.  Tiling in Interactive Panoramic Video: Approaches and Evaluation , 2016, IEEE Transactions on Multimedia.

[10]  David Chu,et al.  FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization , 2017, GETMBL.

[11]  David Chu,et al.  FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization , 2016, MobiSys.

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[14]  Kenny Mitchell,et al.  User, metric, and computational evaluation of foveated rendering methods , 2016, SAP.

[15]  Miska M. Hannuksela,et al.  Viewport-Adaptive Encoding and Streaming of 360-Degree Video for Virtual Reality Applications , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[16]  Chau Yuen,et al.  Enabling Adaptive High-Frame-Rate Video Streaming in Mobile Cloud Gaming Applications , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Y. Dobashi,et al.  Proxy-guided Image-based Rendering for Mobile Devices , 2016 .

[18]  Mahadev Satyanarayanan,et al.  Towards wearable cognitive assistance , 2014, MobiSys.

[19]  Ben H. H. Juurlink,et al.  Parallel Scalability and Efficiency of HEVC Parallelization Approaches , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Alec Wolman,et al.  Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming , 2015, MobiSys.

[21]  Omid Salehi-Abari,et al.  Enabling High-Quality Untethered Virtual Reality , 2017, NSDI.

[22]  Alec Wolman,et al.  Demo: Kahawai: high-quality mobile gaming using GPU offload , 2015, MobiSys.

[23]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[24]  Cheng-Hsin Hsu,et al.  Using graphics rendering contexts to enhance the real-time video coding for mobile cloud gaming , 2011, ACM Multimedia.

[25]  Akira Kojima,et al.  HMD Viewing Spherical Video Streaming System , 2014, ACM Multimedia.

[26]  Xu Chen,et al.  COMET: Code Offload by Migrating Execution Transparently , 2012, OSDI.

[27]  Yunhao Liu,et al.  It starts with iGaze: visual attention driven networking with smart glasses , 2014, MobiCom.

[28]  Weichao Li,et al.  Demystifying and Puncturing the Inflated Delay in Smartphone-based WiFi Network Measurement , 2016, CoNEXT.

[29]  Cheng-Hsin Hsu,et al.  A Survey of Interactive Remote Rendering Systems , 2015, ACM Comput. Surv..

[30]  Ryan Shea,et al.  Cloud gaming: architecture and performance , 2013, IEEE Network.

[31]  Mahadev Satyanarayanan,et al.  Agile application-aware adaptation for mobility , 1997, SOSP.

[32]  Cheng-Hsin Hsu,et al.  Performance Measurements of Virtual Reality Systems: Quantifying the Timing and Positioning Accuracy , 2016, ACM Multimedia.

[33]  Mahadev Satyanarayanan,et al.  Predictive Resource Management for Wearable Computing , 2003, MobiSys '03.

[34]  Ben H. H. Juurlink,et al.  Parallel video decoding in the emerging HEVC standard , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  Giuseppe Bianchi,et al.  Energy consumption anatomy of 802.11 devices and its implication on modeling and design , 2012, CoNEXT '12.