Optimizing HTTP-Based Adaptive Streaming in Vehicular Environment Using Markov Decision Process

Hypertext transfer protocol (HTTP) is the fundamental mechanics supporting web browsing on the Internet. An HTTP server stores large volumes of contents and delivers specific pieces to the clients when requested. There is a recent move to use HTTP for video streaming as well, which promises seamless integration of video delivery to existing HTTP-based server platforms. This is achieved by segmenting the video into many small chunks and storing these chunks as separate files on the server. For adaptive streaming, the server stores different quality versions of the same chunk in different files to allow real-time quality adaptation of the video due to network bandwidth variation experienced by a client. For each chunk of the video, which quality version to download, therefore, becomes a major decision-making challenge for the streaming client, especially in vehicular environment with significant uncertainty in mobile bandwidth. In this paper, we demonstrate that for such decision making, the Markov decision process (MDP) is superior to previously proposed non-MDP solutions. Using publicly available video and bandwidth datasets, we show that the MDP achieves up to a 15x reduction in playback deadline miss compared to a well-known non-MDP solution when the MDP has the prior knowledge of the bandwidth model. We also consider a model-free MDP implementation that uses Q-learning to gradually learn the optimal decisions by continuously observing the outcome of its decision making. We find that the MDP with Q-learning significantly outperforms the MDP that uses bandwidth models.