Block-based Temporal Metric for Video Quality Assessment

Perceptual video quality assessment (VQA) is an integral part of various video applications as it enables quality-control for videos delivered to end-users. Video Multi-method Assessment Fusion (VMAF) was recently proposed as a full reference VQA model that combines quality-aware features to predict perceptual quality. Owing to the limited spatiotemporal resolution capacity of the human eye, VMAF uses motion along with other spatial features to configure its model. The model implements a basic co-located luminance difference method to determine motion. It has been observed that this method is inadequate to capture the temporal characteristic of the video.In this paper, we propose an improvement to the existing temporal metric of VMAF. The newly proposed Temporal Motion Vector based VMAF (TMV-VMAF) replaces the existing temporal metric in VMAF with a block-based motion state classification method that approximates motion score of a frame by leveraging motion estimation and block level energy information. This temporal feature is fed along with other spatial features of VMAF into a Support Vector Regression framework where the model is trained using a database of HD videos. Our results show that the TMV-VMAF achieves better correlation to opinion score when compared to the existing VMAF.

[1]  Wonha Kim,et al.  Video Processing for Human Perceptual Visual Quality-Oriented Video Coding , 2013, IEEE Transactions on Image Processing.

[2]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[3]  Alan C. Bovik,et al.  Fast structural similarity index algorithm , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Paul M. Chau,et al.  Objective Human Visual System Based Video Quality Assessment Metric for Low Bit-Rate Video Communication Systems , 2006, 2006 IEEE Workshop on Multimedia Signal Processing.

[5]  Alan C. Bovik,et al.  Spatiotemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Zhibing Wang,et al.  HVS-based structural similarity for image quality assessment , 2008, 2008 9th International Conference on Signal Processing.

[7]  Fan Zhang,et al.  Image Quality Assessment by Separately Evaluating Detail Losses and Additive Impairments , 2011, IEEE Transactions on Multimedia.

[8]  King Ngi Ngan,et al.  Perceptual sensitivity-based rate control method for high efficiency video coding , 2015, Multimedia Tools and Applications.

[9]  Nabendu Chaki,et al.  Moving Object Detection Approaches, Challenges and Object Tracking , 2014 .

[10]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[11]  A. Bovik.  A VISUAL INFORMATION FIDELITY APPROACH TO VIDEO QUALITY ASSESSMENT , 2005 .

[12]  Patrick Le Callet,et al.  Considering Temporal Variations of Spatial Visual Distortions in Video Quality Assessment , 2009, IEEE Journal of Selected Topics in Signal Processing.

[13]  Marcus Barkowsky,et al.  Analysis of Freely Available Dataset for HDTV including Coding and Transmission Distortions , 2010 .

[14]  Theodore B. Trafalis,et al.  Support vector machine for regression and applications to financial forecasting , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[15]  Margaret H. Pinson,et al.  Temporal Video Quality Model Accounting for Variable Frame Delay Distortions , 2014, IEEE Transactions on Broadcasting.

[16]  Patrick Le Callet,et al.  Suitable methodology in subjective video quality assessment: a resolution dependent paradigm , 2008 .

[17]  Prateek Gupta,et al.  A modified PSNR metric based on HVS for quality assessment of color images , 2011, 2011 International Conference on Communication and Industrial Application.