Supporting visual quality assessment with machine learning

Objective metrics for visual quality assessment often base their reliability on the explicit modeling of the highly non-linear behavior of human perception; as a result, they may be complex and computationally expensive. Conversely, machine learning (ML) paradigms allow to tackle the quality assessment task from a different perspective, as the eventual goal is to mimic quality perception instead of designing an explicit model the human visual system. Several studies already proved the ability of ML-based approaches to address visual quality assessment; nevertheless, these paradigms are highly prone to overfitting, and their overall reliability may be questionable. In fact, a prerequisite for successfully using ML in modeling perceptual mechanisms is a profound understanding of the advantages and limitations that characterize learning machines. This paper illustrates and exemplifies the good practices to be followed.

[1]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[2]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[3]  Pamela C. Cosman,et al.  Modeling packet-loss visibility in MPEG-2 video , 2006, IEEE Transactions on Multimedia.

[4]  P. Gastaldo,et al.  MACHINE LEARNING SOLUTIONS FOR OBJECTIVE VISUAL QUALITY ASSESSMENT , 2011 .

[5]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[6]  Judith Redi,et al.  Circular-ELM for the reduced-reference assessment of perceived image quality , 2013, Neurocomputing.

[7]  Dominik Strohmeier,et al.  Evaluation of differences in quality of experience features for test stimuli of good-only and bad-only overall audiovisual quality , 2013, Electronic Imaging.

[8]  Yang Hu,et al.  Machine Learning to Design Full-reference Image Quality Assessment Algorithm , 2013 .

[9]  LinWeisi,et al.  Objective image quality assessment based on support vector regression , 2010 .

[10]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[11]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[12]  Ashish Kapoor,et al.  Learning a blind measure of perceptual image quality , 2011, CVPR 2011.

[13]  Weisi Lin,et al.  Objective Image Quality Assessment Based on Support Vector Regression , 2010, IEEE Transactions on Neural Networks.

[14]  Cesare Alippi,et al.  Genetic-algorithm programming environments , 1994, Computer.

[15]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[16]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[17]  Yong Huang,et al.  Texture decomposition by harmonics extraction from higher order statistics , 2004, IEEE Trans. Image Process..

[18]  Christophe Charrier,et al.  Machine learning to design full-reference image quality assessment algorithm , 2012, Signal Process. Image Commun..

[19]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[20]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[21]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[22]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[23]  Xiaojun Wu,et al.  Blind Image Quality Assessment Using a General Regression Neural Network , 2011, IEEE Transactions on Neural Networks.

[24]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[25]  Judith Redi,et al.  Color Distribution Information for the Reduced-Reference Assessment of Perceived Image Quality , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Margaret H. Pinson,et al.  Comparing subjective video quality testing methodologies , 2003, Visual Communications and Image Processing.

[27]  Alessandro Sperduti,et al.  Mining Structured Data , 2010, IEEE Computational Intelligence Magazine.

[28]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[29]  Judith Redi,et al.  Comparing subjective image quality measurement methods for the creation of public databases , 2010, Electronic Imaging.

[30]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[31]  Sheila S. Hemami,et al.  No-reference image and video quality estimation: Applications and human-motivated design , 2010, Signal Process. Image Commun..

[32]  Nicu Sebe,et al.  Machine Learning in Computer Vision , 2006, Computational Imaging and Vision.

[33]  Susan Farnand,et al.  Image Quality and System Performance VIII , 2008 .

[34]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[35]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[36]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[37]  John R. Koza,et al.  Genetic programming as a means for programming computers by natural selection , 1994 .

[38]  Francisco Herrera,et al.  A Survey on the Application of Genetic Programming to Classification , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[39]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[40]  Thomas Sporer,et al.  PEAQ - The ITU Standard for Objective Measurement of Perceived Audio Quality , 2000 .

[41]  Christian Viard-Gaudin,et al.  A Convolutional Neural Network Approach for Objective Video Quality Assessment , 2006, IEEE Transactions on Neural Networks.

[42]  Jiong Wang,et al.  On the Convergence of Generalized Simultaneous Iterative Reconstruction Algorithms , 2007, IEEE Transactions on Image Processing.

[43]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[44]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[45]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[46]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[47]  Nikolay N. Ponomarenko,et al.  Color image database for evaluation of image quality metrics , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[48]  Paolo Gastaldo,et al.  Objective quality assessment of MPEG-2 video streams by using CBP neural networks , 2002, IEEE Trans. Neural Networks.

[49]  Russell M. Mersereau,et al.  Rate-quality tradeoff MPEG video encoder , 1999, Signal Process. Image Commun..

[50]  Paolo Gastaldo,et al.  Neural networks for the no-reference assessment of perceived quality , 2005, J. Electronic Imaging.

[51]  Patrick Le Callet,et al.  Subjective quality assessment IRCCyN/IVC database , 2004 .

[52]  Weisi Lin,et al.  Video quality assessment using neural network based on multi-feature extraction , 2003, Visual Communications and Image Processing.

[53]  Azeddine Beghdadi,et al.  Image quality assessment based on distortion identification , 2011, Electronic Imaging.

[54]  S. Tubaro,et al.  Subjective assessment of H.264/AVC video sequences transmitted over a noisy channel , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[55]  Weisi Lin,et al.  Low-Complexity Video Quality Assessment Using Temporal Quality Variations , 2012, IEEE Transactions on Multimedia.

[56]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[57]  Judith Redi,et al.  Efficient neural-network-based no-reference approach to an overall quality metric for JPEG and JPEG2000 compressed images , 2011, J. Electronic Imaging.

[58]  Arthur P. Dempster,et al.  Upper and Lower Probabilities Induced by a Multivalued Mapping , 1967, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[59]  R. Venkatesh Babu,et al.  No-reference image quality assessment using modified extreme learning machine classifier , 2009, Appl. Soft Comput..

[60]  Sun-Ok Gwon University of Texas at Austin의 연구 현황 , 2002 .

[61]  Weisi Lin,et al.  Scalable image quality assessment with 2D mel-cepstrum and machine learning approach , 2012, Pattern Recognit..

[62]  Piet Demeester,et al.  Constructing a No-Reference H.264/AVC Bitstream-Based Video Quality Metric Using Genetic Programming-Based Symbolic Regression , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[63]  Yanqing Zhang,et al.  Guest Editors' Introduction to the Special Section: Computational Intelligence Approaches in Computational Biology and Bioinformatics , 2007, IEEE ACM Trans. Comput. Biol. Bioinform..

[64]  Sandro Ridella,et al.  Circular backpropagation networks for classification , 1997, IEEE Trans. Neural Networks.

[65]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[66]  Alexander Raake,et al.  No-reference video quality assessment for SD and HD H.264/AVC sequences based on continuous estimates of packet loss visibility , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[67]  Yoshua Bengio,et al.  Exploring Strategies for Training Deep Neural Networks , 2009, J. Mach. Learn. Res..

[68]  Weisi Lin,et al.  SVD-Based Quality Metric for Image and Video Using Machine Learning , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[69]  Driss Aboutajdine,et al.  Video Quality assessment Measure with a Neural Network , 2011 .

[70]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.