High quality, low latency in-home streaming of multimedia applications for mobile devices

Today, mobile devices like smartphones and tablets are becoming more powerful and exhibit enhanced 3D graphics performance. However, the overall computing power of these devices is still limited regarding usage scenarios like photo-realistic gaming, enabling an immersive virtual reality experience or real-time processing and visualization of big data. To overcome these limitations application streaming solutions are constantly gaining focus. The idea is to transfer the graphics output of an application running on a server or even a cluster to a mobile device, conveying the impression that the application is running locally. User inputs on the mobile client side are processed and sent back to the server. The main criteria for successful application streaming are low latency, since users want to interact with the scene in near real-time, as well as high image quality. Here, we present a novel application framework suitable for streaming applications from high-end machines to mobile devices. Using real-time ETC1 compression in combination with a distributed rendering architecture we fully leverage recent progress in wireless computer networking standards (IEEE 802.11ac) for mobile devices, achieving much higher image quality at half the latency compared to other inhome streaming solutions.

[1]  I. Nave,et al.  Games@large graphics streaming architecture , 2008, 2008 IEEE International Symposium on Consumer Electronics.

[2]  Sabine C. Mueller,et al.  BALL - biochemical algorithms library 1.3 , 2010, BMC Bioinformatics.

[3]  Tomas Akenine-Möller,et al.  iPACKMAN: high-quality, low-complexity texture compression for mobile phones , 2005, HWWS '05.

[4]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[5]  Andy Hopper,et al.  Virtual Network Computing , 1998, IEEE Internet Comput..

[6]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[7]  Cheng-Hsin Hsu,et al.  GamingAnywhere: an open cloud gaming system , 2013, MMSys.

[8]  Hans-Peter Lenhof,et al.  BALLView: a tool for research and education in molecular modeling , 2006, Bioinform..

[9]  Philipp Slusallek,et al.  Networked Displays for VR Applications: Display as a Service , 2012, ICAT/EGVE/EuroVR.

[10]  Erik Reinhard,et al.  Parallel and Distributed Photo-Realistic Rendering , 1998 .

[11]  Yao Wang,et al.  Video Processing and Communications , 2001 .

[12]  Philipp Slusallek,et al.  Real-Time Ray Tracing of Complex Molecular Scenes , 2010, 2010 14th International Conference Information Visualisation.

[13]  Geoffrey C. Fox,et al.  Parallel Computing Works , 1994 .

[14]  Marc Levoy,et al.  Parallel visualization algorithms: performance and architectural implications , 1994, Computer.

[15]  Ingo Wald,et al.  Embree ray tracing kernels for CPUs and the Xeon Phi architecture , 2013, SIGGRAPH '13.

[16]  Daniel Pohl,et al.  Improved pre-warping for wide angle, head mounted displays , 2013, VRST '13.

[17]  Wang Yu-ying Remote Desktop Connection , 2006 .

[18]  Truong Q. Nguyen,et al.  Quality Enhancement for Motion JPEG using Temporal Redundancies , 2007, 2007 IEEE International Conference on Image Processing.