Blind Natural Video Quality Prediction via Statistical Temporal Features and Deep Spatial Features

Due to the wide range of different natural temporal and spatial distortions appearing in user generated video content, blind assessment of natural video quality is a challenging research problem. In this study, we combine the hand-crafted statistical temporal features used in a state-of-the-art video quality model and spatial features obtained from convolutional neural network trained for image quality assessment via transfer learning. Experimental results on two recently published natural video quality databases show that the proposed model can predict subjective video quality more accurately than the publicly available video quality models representing the state-of-the-art. The proposed model is also competitive in terms of computational complexity.

[1]  Paolo Napoletano,et al.  On the use of deep learning for blind image quality assessment , 2016, Signal Image Video Process..

[2]  Alan Conrad Bovik,et al.  Large-Scale Study of Perceptual Video Quality , 2018, IEEE Transactions on Image Processing.

[3]  Jari Korhonen,et al.  Two-Level Approach for No-Reference Consumer Video Quality Assessment , 2019, IEEE Transactions on Image Processing.

[4]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Tone-Mapped HDR Pictures , 2017, IEEE Transactions on Image Processing.

[5]  Alan C. Bovik,et al.  UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content , 2020, IEEE Transactions on Image Processing.

[6]  Mikko Nuutinen,et al.  CVD2014—A Database for Evaluating No-Reference Video Quality Assessment Algorithms , 2016, IEEE Transactions on Image Processing.

[7]  Balu Adsumilli,et al.  YouTube UGC Dataset for Video Compression Research , 2019, 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP).

[8]  Alan C. Bovik,et al.  Perceptual quality prediction on authentically distorted images using a bag of features approach , 2016, Journal of vision.

[9]  Alan C. Bovik,et al.  No Reference Video Quality Assessment with authentic distortions using 3-D Deep Convolutional Neural Network , 2020 .

[10]  Lai-Man Po,et al.  No-Reference Video Quality Assessment With 3D Shearlet Transform and Convolutional Neural Networks , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Vijayan K. Asari,et al.  No-Reference Video Quality Assessment Based on Artifact Measurement and Statistical Analysis , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Dietmar Saupe,et al.  Deeprn: A Content Preserving Deep Architecture for Blind Image Quality Assessment , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[13]  Dietmar Saupe,et al.  Comment on "No-Reference Video Quality Assessment Based on the Temporal Pooling of Deep Features" , 2020, ArXiv.

[14]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[15]  Lei Zhang,et al.  A Probabilistic Quality Representation Approach to Deep Blind Image Quality Prediction , 2017, ArXiv.

[16]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Domonkos Varga,et al.  No-Reference Video Quality Assessment Based on the Temporal Pooling of Deep Features , 2019, Neural Processing Letters.

[19]  Alan C. Bovik,et al.  A Completely Blind Video Integrity Oracle , 2016, IEEE Transactions on Image Processing.

[20]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[21]  Alan C. Bovik,et al.  Large Scale Subjective Video Quality Study , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[22]  Zhengfang Duanmu,et al.  End-to-End Blind Quality Assessment of Compressed Videos Using Deep Neural Networks , 2018, ACM Multimedia.

[23]  Alan C. Bovik,et al.  In-Capture Mobile Video Distortions: A Study of Subjective Behavior and Objective Algorithms , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Cordelia Schmid,et al.  Analysing Domain Shift Factors between Videos and Images for Object Detection , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Dietmar Saupe,et al.  Spatiotemporal Feature Combination Model for No-Reference Video Quality Assessment , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[26]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[27]  Ming Jiang,et al.  Quality Assessment of In-the-Wild Videos , 2019, ACM Multimedia.

[28]  Stefan Winkler,et al.  The Evolution of Video Quality Measurement: From PSNR to Hybrid Metrics , 2008, IEEE Transactions on Broadcasting.

[29]  Dietmar Saupe,et al.  No-Reference Video Quality Assessment using Multi-Level Spatially Pooled Features , 2019, ArXiv.

[30]  Dietmar Saupe,et al.  The Konstanz natural video database (KoNViD-1k) , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[31]  Alan C. Bovik,et al.  Massive Online Crowdsourced Study of Subjective and Objective Picture Quality , 2015, IEEE Transactions on Image Processing.

[32]  Jacob Søgaard,et al.  No-Reference Video Quality Assessment Using Codec Analysis , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  J ValdésJulio,et al.  2006 Special issue , 2006 .

[34]  Christophe Charrier,et al.  Blind Prediction of Natural Video Quality , 2014, IEEE Transactions on Image Processing.

[35]  Dietmar Saupe,et al.  KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment , 2019, IEEE Transactions on Image Processing.

[36]  Xuelong Li,et al.  Spatiotemporal Statistics for Video Quality Assessment , 2016, IEEE Transactions on Image Processing.