Learning to Rank for Blind Image Quality Assessment

Blind image quality assessment (BIQA) aims to predict perceptual image quality scores without access to reference images. State-of-the-art BIQA methods typically require subjects to score a large number of images to train a robust model. However, subjective quality scores are imprecise, biased, and inconsistent, and it is challenging to obtain a large-scale database, or to extend existing databases, because of the inconvenience of collecting images, training the subjects, conducting subjective experiments, and realigning human quality evaluations. To combat these limitations, this paper explores and exploits preference image pairs (PIPs) such as the quality of image Ia is better than that of image Ib for training a robust BIQA model. The preference label, representing the relative quality of two images, is generally precise and consistent, and is not sensitive to image content, distortion type, or subject identity; such PIPs can be generated at a very low cost. The proposed BIQA method is one of learning to rank. We first formulate the problem of learning the mapping from the image features to the preference label as one of classification. In particular, we investigate the utilization of a multiple kernel learning algorithm based on group lasso to provide a solution. A simple but effective strategy to estimate perceptual image quality scores is then presented. Experiments show that the proposed BIQA method is highly effective and achieves a performance comparable with that of state-of-the-art BIQA algorithms. Moreover, the proposed method can be easily extended to new distortion categories.

[1]  Alan C. Bovik,et al.  Objective quality assessment of multiply distorted images , 2012, 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[2]  Dewen Hu,et al.  Scene recognition combining structural and textural features , 2011, Science China Information Sciences.

[3]  Xuelong Li,et al.  Rank Preserving Sparse Learning for Kinect Based Scene Classification , 2013, IEEE Transactions on Cybernetics.

[4]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[5]  David S. Doermann,et al.  Real-Time No-Reference Image Quality Assessment Based on Filter Learning , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Xinbo Gao,et al.  Saliency-Guided Deep Framework for Image Quality Assessment , 2015, IEEE MultiMedia.

[7]  Xiaojun Wu,et al.  Blind Image Quality Assessment Using a General Regression Neural Network , 2011, IEEE Transactions on Neural Networks.

[8]  Aimin Hao,et al.  Multi-scale local features based on anisotropic heat diffusion and global eigen-structure , 2012, Science China Information Sciences.

[9]  Maya R. Gupta,et al.  How to Analyze Paired Comparison Data , 2011 .

[10]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[11]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[12]  Jun Yu,et al.  Click Prediction for Web Image Reranking Using Multimodal Sparse Coding , 2014, IEEE Transactions on Image Processing.

[13]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[14]  Hang Li,et al.  A Short Introduction to Learning to Rank , 2011, IEICE Trans. Inf. Syst..

[15]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[16]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[18]  Xuelong Li,et al.  Blind Image Quality Assessment via Deep Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Xuelong Li,et al.  Image esthetic assessment using both hand-crafting and semantic features , 2014, Neurocomputing.

[20]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[21]  Franco Scarselli,et al.  SortNet: Learning to Rank by a Neural Preference Function , 2011, IEEE Transactions on Neural Networks.

[22]  Huib de Ridder,et al.  Current issues and new techniques in visual quality assessment , 1996, ICIP.

[23]  Nikolay N. Ponomarenko,et al.  Color image database TID2013: Peculiarities and preliminary results , 2013, European Workshop on Visual Information Processing (EUVIP).

[24]  Xuelong Li,et al.  Transductive Face Sketch-Photo Synthesis , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Patrick Le Callet,et al.  Tradeoffs in subjective testing methods for image and video quality assessment , 2010, Electronic Imaging.

[26]  Meng Wang,et al.  Image clustering based on sparse patch alignment framework , 2014, Pattern Recognit..

[27]  Xuelong Li,et al.  Sparse representation for blind image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Thore Graepel,et al.  Large Margin Rank Boundaries for Ordinal Regression , 2000 .

[29]  Xuelong Li,et al.  Scene Parsing From an MAP Perspective , 2015, IEEE Transactions on Cybernetics.

[30]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[31]  Chih-Jen Lin,et al.  Large-Scale Linear RankSVM , 2014, Neural Computation.

[32]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[33]  Hongyuan Zha,et al.  A General Boosting Method and its Application to Learning Ranking Functions for Web Search , 2007, NIPS.

[34]  Joydeep Ghosh,et al.  Blind Image Quality Assessment Without Human Training Using Latent Quality Factors , 2012, IEEE Signal Processing Letters.

[35]  S. Sathiya Keerthi,et al.  Efficient algorithms for ranking with SVMs , 2010, Information Retrieval.

[36]  Ivor W. Tsang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Soft Margin Multiple Kernel Learning , 2022 .

[37]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[38]  David S. Doermann,et al.  No-Reference Image Quality Assessment Using Visual Codebooks , 2012, IEEE Transactions on Image Processing.

[39]  Eyke Hüllermeier,et al.  Label ranking by learning pairwise preferences , 2008, Artif. Intell..

[40]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[41]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.

[42]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[43]  Cong Li,et al.  A Unifying Framework for Typical Multitask Multiple Kernel Learning Problems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Xuelong Li,et al.  Universal Blind Image Quality Assessment Metrics Via Natural Scene Statistics and Multiple Kernel Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[46]  Xuelong Li,et al.  Error Analysis of Stochastic Gradient Descent Ranking , 2013, IEEE Transactions on Cybernetics.

[47]  Zenglin Xu,et al.  Simple and Efficient Multiple Kernel Learning by Group Lasso , 2010, ICML.

[48]  Weisi Lin,et al.  Image Quality Assessment Based on Gradient Similarity , 2012, IEEE Transactions on Image Processing.

[49]  Alan C. Bovik,et al.  Making image quality assessment robust , 2012, 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[50]  Lei Zhang,et al.  Learning without Human Scores for Blind Image Quality Assessment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  L. Thurstone A law of comparative judgment. , 1994 .

[52]  Pirkko Oittinen,et al.  Naturalness and interestingness of test images for visual quality evaluation , 2011, Electronic Imaging.

[53]  Ashish Kapoor,et al.  Learning a blind measure of perceptual image quality , 2011, CVPR 2011.

[54]  Nello Cristianini,et al.  A statistical framework for genomic data fusion , 2004, Bioinform..

[55]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[56]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[57]  Y. Rui,et al.  Learning to Rank Using User Clicks and Visual Features for Image Retrieval , 2015, IEEE Transactions on Cybernetics.

[58]  Xuelong Li,et al.  Ranking Graph Embedding for Learning to Rerank , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[59]  A Parducci,et al.  The category effect with rating scales: number of categories, number of stimuli, and method of presentation. , 1986, Journal of experimental psychology. Human perception and performance.