Is It Possible to Predict Human Perception of Video Quality? The Assessment of Sencogi Quality Metric

Sencogi Quality Metric (SenQM) is a novel objective metric for video quality assessment. SenQM infers video quality scores from the spatio-temporal evolution of videos. The quality model behind SenQM is based on an algorithm developed by Cogisen for modelling dynamic phenomena generated by complex systems. In the field of video compression, Cogisen’s algorithm uses machine learning to model human perception of video quality by extracting meaningful information directly from the video data domain and its frequency representation. The model has been trained over datasets of (i) x264 compressed videos as input data and (ii) the corresponding subjective Mean Opinion Scores as ground truth. This study introduces the model behind SenQM and how the proposed metric performs in subjective video quality prediction compared to the most used video quality assessment methods, i.e. PSNR, SSIM, and Netflix’s VMAF. Results indicate a significantly higher prediction performance in terms of monotonicity, consistency, and accuracy than the compared metrics. SenQM quality scores show significantly higher variations for 352 × 288 resolution videos with equivalent levels of degradation, and outstands PSNR, SSIM, and VMAF in predicting subjective scores of increasing levels of compression without being affected by either the degradation level or the video content.

[1]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Yo-Sung Ho,et al.  Sparse Representation-Based Video Quality Assessment for Synthesized 3D Videos , 2020, IEEE Transactions on Image Processing.

[3]  Maria Laura Mele,et al.  Using Spatio-Temporal Saliency to Predict Subjective Video Quality: A New High-Speed Objective Assessment Metric , 2017, HCI.

[4]  Yun Zhang,et al.  Machine learning based video coding optimizations: A survey , 2020, Inf. Sci..

[5]  Silvia Colabrese,et al.  The Assessment of Sencogi: A Visual Complexity Model Predicting Visual Fixations , 2019, HCI.

[6]  L. Itti,et al.  Top-down influences on visual attention during listening are modulated by observer sex , 2012, Vision Research.

[7]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[8]  Snjezana Rimac-Drlje,et al.  Foveated mean squared error—a novel video quality metric , 2010, Multimedia Tools and Applications.

[9]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[10]  Qionghai Dai,et al.  Full-Reference Quality Assessment of Stereoscopic Images by Learning Binocular Receptive Field Properties , 2015, IEEE Transactions on Image Processing.

[11]  Philip J. Corriveau,et al.  The Video Quality Experts Group: Evaluates Objective Methods of Video Image Quality Assessment , 1998 .

[12]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[13]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[14]  Peter Lambert,et al.  Assessing Quality of Experience of IPTV and Video on Demand Services in Real-Life Environments , 2010, IEEE Transactions on Broadcasting.

[15]  Snjezana Rimac-Drlje,et al.  Review of objective video quality metrics and performance comparison using different databases , 2013, Signal Process. Image Commun..

[16]  Maria Laura Mele,et al.  Sencogi Spatio-Temporal Saliency: A New Metric for Predicting Subjective Video Quality on Mobile Devices , 2018, HCI.

[17]  Xinbo Gao,et al.  Objective Video Quality Assessment Combining Transfer Learning With CNN , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Maria Laura Mele,et al.  The Web-based Subjective Quality Assessment of an Adaptive Image Compression Plug-in , 2017, VISIGRAPP.

[19]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[20]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.