Video quality assessment using content-weighted spatial and temporal pooling method

Abstract. Video quality assessment plays an important role in video processing and communication applications. We propose a full reference video quality metric by combining a content-weighted spatial pooling strategy with a temporal pooling strategy. All pixels in a frame are classified into edge, texture, and smooth regions, and their structural similarity image index (SSIM) maps are divided into increasing and saturated regions by the curve of their SSIM values, then a content weight method is applied to increasing regions to get the score of an image frame. Finally, a temporal pooling method is used to get the overall video quality. Experimental results on the LIVE and IVP video quality databases show our proposed method works well in matching subjective scores.

[1]  Vijayan K. Asari,et al.  No-Reference Video Quality Assessment Based on Artifact Measurement and Statistical Analysis , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Alan C. Bovik,et al.  Fast algorithms for foveated video processing , 2003, IEEE Trans. Circuits Syst. Video Technol..

[3]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[4]  Chaofeng Li,et al.  Content-weighted video quality assessment using a three-component image model , 2010, J. Electronic Imaging.

[5]  Jordan W. Suchow,et al.  Motion Silences Awareness of Visual Change , 2011, Current Biology.

[6]  Marios S. Pattichis,et al.  Foveated video compression with optimal rate control , 2001, IEEE Trans. Image Process..

[7]  King Ngi Ngan,et al.  Full-Reference Video Quality Assessment by Decoupling Detail Losses and Additive Impairments , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Mark D. Fairchild,et al.  Color Appearance Models , 1997, Computer Vision, A Reference Guide.

[9]  Sheila S. Hemami,et al.  A metric for continuous quality evaluation of compressed video with severe distortions , 2004, Signal Process. Image Commun..

[10]  Michael Yuen,et al.  A survey of hybrid MC/DPCM/DCT video coding distortions , 1998, Signal Process..

[11]  Sheila S. Hemami,et al.  A scalable wavelet-based video distortion metric and applications , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Tian-Sheuan Chang,et al.  Analysis of color space and similarity measure impact on stereo block matching , 2008, APCCAS 2008 - 2008 IEEE Asia Pacific Conference on Circuits and Systems.

[13]  Alan C. Bovik,et al.  Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos , 2010, IEEE Transactions on Image Processing.

[14]  Patrick Le Callet,et al.  On the performance of human visual system based image quality assessment metric using wavelet domain , 2008, Electronic Imaging.

[15]  Alan C. Bovik,et al.  High quality, low delay foveated visual communications over mobile channels , 2005, J. Vis. Commun. Image Represent..

[16]  Chaofeng Li,et al.  Content-partitioned structural similarity index for image quality assessment , 2010, Signal Process. Image Commun..

[17]  Touradj Ebrahimi,et al.  Balancing Attended and Global Stimuli in Perceived Video Quality Assessment , 2011, IEEE Transactions on Multimedia.

[18]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[19]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[20]  Marios S. Pattichis,et al.  Foveated video quality assessment , 2002, IEEE Trans. Multim..

[21]  Garrett Johnson.  Color and Image Appearance Models , 2008 .

[22]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[23]  Ranjit Singh,et al.  State of the art and research issues in video quality assessment , 2014, 2014 Recent Advances in Engineering and Computational Sciences (RAECS).

[24]  Margaret H. Pinson,et al.  Temporal Video Quality Model Accounting for Variable Frame Delay Distortions , 2014, IEEE Transactions on Broadcasting.

[25]  Alan C. Bovik,et al.  Video Quality Pooling Adaptive to Perceptual Distortion Severity , 2013, IEEE Transactions on Image Processing.

[26]  Weisi Lin,et al.  Low-Complexity Video Quality Assessment Using Temporal Quality Variations , 2012, IEEE Transactions on Multimedia.

[27]  Alan C. Bovik,et al.  Spatio-temporal quality pooling accounting for transient severe impairments and egomotion , 2011, 2011 18th IEEE International Conference on Image Processing.

[28]  Patrick Le Callet,et al.  Considering Temporal Variations of Spatial Visual Distortions in Video Quality Assessment , 2009, IEEE Journal of Selected Topics in Signal Processing.

[29]  Rajiv Soundararajan,et al.  Video Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Eero P. Simoncelli,et al.  Noise characteristics and prior expectations in human visual speed perception , 2006, Nature Neuroscience.

[31]  Christophe Charrier,et al.  Blind Prediction of Natural Video Quality , 2014, IEEE Transactions on Image Processing.