Combining audio and video metrics to assess audio-visual quality

In this work, we studied the use of combination models to integrate audio and video quality estimates to predict the overall audio-visual quality. More specifically, an overall quality prediction for an audio-visual signal is obtained by combining the outputs of individual audio and video quality metrics with either a linear, a Minkowski, or a power function. A total of 7 different video quality metrics are considered, from which 3 are Full-Reference and 4 are No-Reference. Similarly, a total of 4 audio quality metrics are tested, 2 of which are Full-Reference and 2 are No-Reference. In total, we tested 18 Full-Reference audio-visual combination metrics and 24 No-Reference audio-visual combination metrics. The performance of all combination metrics are tested on two different audio-visual databases. Therefore, besides analysing the performance of a set of individual audio and video quality metrics, we analyzed the performance of the models that combine these audio and video quality metrics. This work gives an important contribution to the area of audio-visual quality assessment, since previous works either tested combination models only on subjective quality scores or used linear models to combine the outputs of a limited number of audio and video quality metrics.

[1]  Sanjit K. Mitra,et al.  No-reference video quality metric based on artifact measurements , 2005, IEEE International Conference on Image Processing 2005.

[2]  Alexander Raake,et al.  Impairment-Factor-Based Audiovisual Quality Model for IPTV: Influence of Video Resolution, Degradation Type, and Content Type , 2011, EURASIP J. Image Video Process..

[3]  Alan C. Bovik,et al.  Blind/Referenceless Image Spatial Quality Evaluator , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[4]  Andrew Hines,et al.  Measuring and monitoring speech quality for voice over IP with POLQA, viSQOL and p.563 , 2015, INTERSPEECH.

[5]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[6]  Margaret H. Pinson,et al.  Audiovisual Quality Components , 2011, IEEE Signal Processing Magazine.

[7]  J. Berger,et al.  P.563—The ITU-T Standard for Single-Ended Speech Quality Assessment , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[8]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[9]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[10]  Mylène C. Q. Farias,et al.  A no-reference audio-visual video quality metric , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[11]  Alan C. Bovik,et al.  Visual quality assessment algorithms: what does the future hold? , 2010, Multimedia Tools and Applications.

[12]  Marcus Barkowsky,et al.  Subjective and objective evaluation of an audiovisual subjective dataset for research and development , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[13]  H.-J. Zepernick,et al.  Perceptual-based Quality Metrics for Image and Video Services: A Survey , 2007, 2007 Next Generation Internet Networks.

[14]  Zhou Wang,et al.  No-reference perceptual quality assessment of JPEG compressed images , 2002, Proceedings. International Conference on Image Processing.

[15]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[16]  Anil C. Kokaram,et al.  ViSQOL: The Virtual Speech Quality Objective Listener , 2012, IWAENC.

[17]  Mylène C. Q. Farias,et al.  Full-reference audio-visual video quality metric , 2014, J. Electronic Imaging.

[18]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[19]  Thomas Sporer,et al.  PEAQ - The ITU Standard for Objective Measurement of Perceived Audio Quality , 2000 .

[20]  Shan Gao,et al.  Light-weight audiovisual quality assessment of mobile video: ITU-T Rec. P.1201.1 , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[21]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[22]  D. S. Hands,et al.  A basic multimedia quality model , 2004, IEEE Transactions on Multimedia.

[23]  Bee Ee Khoo,et al.  Objective blur assessment based on contraction errors of local contrast maps , 2015, Multimedia Tools and Applications.

[24]  Stefan Winkler,et al.  Perceived Audiovisual Quality of Low-Bitrate Multimedia Content , 2006, IEEE Transactions on Multimedia.

[25]  Anil C. Kokaram,et al.  Robustness of speech quality metrics to background noise and network degradations: Comparing ViSQOL, PESQ and POLQA , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.