RT-VQM: real-time video quality assessment for adaptive video streaming using GPUs

Adaptive streaming systems gain rising relevance for streaming services. Therefore, the same video is offered in multiple quality versions to clients for adaptation during playback. However, optimizing adaptation in a Quality of Experience (QoE) centric way is difficult. Current systems maximize bit rate, ignoring that different types of adaptation (resolution, framerate, quantization) correlate differently and in a non-linear way with user's perception. User validated video quality metrics can provide precise quality information. However, measurements of state-of-the-art metrics show either high computational intensity or weak correlation with subjective tests. This makes large-scale offline quality assessment processing intensive while real-time constrained scenarios like live streaming and video conferencing are hardly supportable. Consequently, this work presents the Real-Time Video Quality Metric (RT-VQM), a real-time, Graphics Processing Unit (GPU) supported version of the widely used Video Quality Metric (VQM). RT-VQM introduces efficient filtering operations, hardware-supported scaling and high-performance feature pooling. The approach outperforms VQM by a factor of 30, thus enabling a real-time assessment of up to 9 parallel video stream representations up to High Definition (HD) 720 resolution at 30fps.