Resource allocation for cloud-based free viewpoint video rendering for mobile phones

Free viewpoint video (FVV) / Free viewpoint TV (FTV) on mobile devices over cellular networks is very challenging due to the requirement for large bandwidth and limitations in computation and battery life on mobile phones. To address such challenges, in this paper we propose a cloud-based FVV / FTV rendering framework for mobile devices over cellular networks. In this framework, cloud performs rendering for mobile devices. In order to achieve maximum QoE (Quality of Experience) for mobile users, we propose a novel resource allocation scheme, which jointly considers rendering allocation between cloud and client based on user's QoE and rate allocation among texture, depth, and channel rate based on rate-distortion analysis. We formulate this resource allocation scheme as an optimization problem which can then be transformed into a convex optimization for the given rate ratio. Experimental results demonstrate that the proposed cloud-based FVV rendering solution can substantially improve video quality on mobile devices comparing with traditional approaches.