LevelUp: A thin-cloud approach to game livestreaming

Game livestreaming is hugely popular and growing. Each month, Twitch hosts over two million unique broadcasters with a collective audience of 140 million unique viewers. Despite its success, livestreaming services are costly to run. AWS and Azure both charge hundreds of dollars to encode 100 hours of multi-bitrate video, and potentially thousands each month to transfer the video data of one gamer to a relatively small audience. In this work, we demonstrate that mobile edge devices are ready to play a more central role in multi-bitrate livestream- ing. In particular, we explore a new strategy for game livestream- ing that we call a thin-cloud approach. Under a thin-cloud approach, livestreaming services rely on commodity web infrastructure to store and distribute video content and lever- age hardware acceleration on edge devices to transcode video and boost the video quality of low-bitrate streams. We have built a prototype system called LevelUp that embodies the thin-cloud approach, and using our prototype we demonstrate that mobile hardware acceleration can support real- time video transcoding and significantly boost the quality of low-bitrate video through a machine-learning technique called super resolution. We show that super-resolution can improve the visual quality of low-resolution game streams by up to 88% while requiring approximately half the band- width of higher-bitrate streams. Finally, energy experiments show that LevelUp clients consume only 5% of their battery capacity watching 30 minutes of video.

[1]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[2]  Alec Wolman,et al.  Demo: Kahawai: high-quality mobile gaming using GPU offload , 2015, MobiSys.

[3]  Fan Zhang,et al.  Image Quality Assessment by Separately Evaluating Detail Losses and Additive Impairments , 2011, IEEE Transactions on Multimedia.

[4]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Dongsu Han,et al.  Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning , 2020, SIGCOMM.

[6]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[7]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Pan Hu,et al.  Dejavu: Enhancing Videoconferencing with Prior Knowledge , 2019, HotMobile.

[9]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Chih-Yuan Yang,et al.  Single-Image Super-Resolution: A Benchmark , 2014, ECCV.

[11]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Hongzi Mao,et al.  Neural Adaptive Video Streaming with Pensieve , 2017, SIGCOMM.

[13]  Junchen Jiang,et al.  Bridging the Edge-Cloud Barrier for Real-time Advanced Vision Analytics , 2019, HotCloud.

[14]  Geoffrey M. Voelker,et al.  Sprocket: A Serverless Video Processing Framework , 2018, SoCC.

[15]  Jinwoo Shin,et al.  Neural Adaptive Content-aware Internet Video Delivery , 2018, OSDI.

[16]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Keith Winstein,et al.  Salsify: Low-Latency Network Video through Tighter Integration between a Video Codec and a Transport Protocol , 2018, NSDI.

[18]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  Anirudh Sivaraman,et al.  Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads , 2017, NSDI.

[20]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.