Enhancing Temporal Quality Measurements in a Globally Deployed Streaming Video Quality Predictor

Most successful perceptual video quality assessment models are either frame-based, or perform spatiotemporal filtering or motion estimation to model the temporal aspects of video distortions. While good results are obtained on video quality databases, their increased computational complexity often causes video quality engineers to instead rely on simpler image-based quality algorithms. Towards balancing demands between prediction accuracy and compute efficiency, Netflix developed the Video Multi-method Assessment Fusion (VMAF) Framework, an efficient feature-based system that combines multiple perception-based elementary image measurements to produce video quality predictions. However, the current version of VMAF only weakly captures temporal video features which are sensitive to perceptual temporal video distortions. To this end, we propose an enhanced model we call SpatioTemporal VMAF (ST- VMAF) that incorporates temporal features that are easy to compute. We demonstrate the improved performance of ST- VMAF on many subjective video databases. The proposed model will be made available as part of the open source package in https://github.com/Netflix/vmaf.