An Efficient Method for No-Reference Video Quality Assessment

Methods for No-Reference Video Quality Assessment (NR-VQA) of consumer-produced video content are largely investigated due to the spread of databases containing videos affected by natural distortions. In this work, we design an effective and efficient method for NR-VQA. The proposed method exploits a novel sampling module capable of selecting a predetermined number of frames from the whole video sequence on which to base the quality assessment. It encodes both the quality attributes and semantic content of video frames using two lightweight Convolutional Neural Networks (CNNs). Then, it estimates the quality score of the entire video using a Support Vector Regressor (SVR). We compare the proposed method against several relevant state-of-the-art methods using four benchmark databases containing user generated videos (CVD2014, KoNViD-1k, LIVE-Qualcomm, and LIVE-VQC). The results show that the proposed method at a substantially lower computational cost predicts subjective video quality in line with the state of the art methods on individual databases and generalizes better than existing methods in cross-database setup.

[1]  Alan Conrad Bovik,et al.  Large-Scale Study of Perceptual Video Quality , 2018, IEEE Transactions on Image Processing.

[2]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[3]  Alan C. Bovik,et al.  Blind quality assessment of videos using a model of natural scene statistics and motion coherency , 2012, 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[4]  Dietmar Saupe,et al.  The Konstanz natural video database (KoNViD-1k) , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[5]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[6]  Sanjeev Khudanpur,et al.  Deep Neural Network Embeddings for Text-Independent Speaker Verification , 2017, INTERSPEECH.

[7]  David A. Shamma,et al.  YFCC100M , 2015, Commun. ACM.

[8]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Mikko Nuutinen,et al.  CVD2014—A Database for Evaluating No-Reference Video Quality Assessment Algorithms , 2016, IEEE Transactions on Image Processing.

[10]  Mikko Nuutinen,et al.  CID2013: A Database for Evaluating No-Reference Image Quality Assessment Algorithms , 2015, IEEE Transactions on Image Processing.

[11]  Yue Wang,et al.  UGC-VIDEO: Perceptual Quality Assessment of User-Generated Videos , 2019, 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR).

[12]  David S. Doermann,et al.  No-reference video quality assessment via feature learning , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[13]  Paolo Napoletano,et al.  Benchmark Analysis of Representative Deep Neural Network Architectures , 2018, IEEE Access.

[14]  Li Su,et al.  COME for No-Reference Video Quality Assessment , 2018, 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR).

[15]  Kede Ma,et al.  Perceptual Quality Assessment of Smartphone Photography , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[17]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[18]  Jari Korhonen,et al.  Two-Level Approach for No-Reference Consumer Video Quality Assessment , 2019, IEEE Transactions on Image Processing.

[19]  Guangming Shi,et al.  Blind image quality assessment with semantic information , 2019, J. Vis. Commun. Image Represent..

[20]  Neil Birkbeck,et al.  Subjective Quality Assessment For Youtube Ugc Dataset , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[21]  Mounir Kaaniche,et al.  Towards a video quality assessment based framework for enhancement of laparoscopic videos , 2020, Medical Imaging.

[22]  Paolo Napoletano,et al.  On the use of deep learning for blind image quality assessment , 2016, Signal Image Video Process..

[23]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[24]  Raimondo Schettini,et al.  No-Reference Quality Assessment of In-Capture Distorted Videos , 2020, J. Imaging.

[25]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[26]  Ming Jiang,et al.  Quality Assessment of In-the-Wild Videos , 2019, ACM Multimedia.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Sumohana S. Channappayya,et al.  No-Reference Video Quality Assessment Using Natural Spatiotemporal Scene Statistics , 2020, IEEE Transactions on Image Processing.

[29]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[30]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Pengfei Chen,et al.  RIRNet: Recurrent-In-Recurrent Network for Video Quality Assessment , 2020, ACM Multimedia.

[32]  Lai-Man Po,et al.  No-Reference Video Quality Assessment With 3D Shearlet Transform and Convolutional Neural Networks , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Alan C. Bovik,et al.  Perceptual quality prediction on authentically distorted images using a bag of features approach , 2016, Journal of vision.

[34]  Joshua Peter Ebenezer,et al.  No-Reference Video Quality Assessment Using Space-Time Chips , 2020, 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP).

[35]  Judith Redi,et al.  Semantic-aware blind image quality assessment , 2018, Signal Process. Image Commun..

[36]  Alan C. Bovik,et al.  In-Capture Mobile Video Distortions: A Study of Subjective Behavior and Objective Algorithms , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Shaoping Xu,et al.  Blind Image Quality Assessment Based on Multi-Window Method and HSV Color Space , 2019, Applied Sciences.

[38]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Tone-Mapped HDR Pictures , 2017, IEEE Transactions on Image Processing.

[39]  Xuelong Li,et al.  Image quality assessment and human visual system , 2010, Visual Communications and Image Processing.

[40]  Tingting Jiang,et al.  Unified Quality Assessment of in-the-Wild Videos with Mixed Datasets Training , 2021, Int. J. Comput. Vis..

[41]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[42]  Junyong You,et al.  Blind Natural Video Quality Prediction via Statistical Temporal Features and Deep Spatial Features , 2020, ACM Multimedia.

[43]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.