SUR-Net: Predicting the Satisfied User Ratio Curve for Image Compression with Deep Learning

The Satisfied User Ratio (SUR) curve for a lossy image compression scheme, e.g., JPEG, characterizes the probability distribution of the Just Noticeable Difference (JND) level, the smallest distortion level that can be perceived by a subject. We propose the first deep learning approach to predict such SUR curves. Instead of the direct approach of regressing the SUR curve itself for a given reference image, our model is trained on pairs of images, original and compressed. Relying on a Siamese Convolutional Neural Network (CNN), feature pooling, a fully connected regression-head, and transfer learning, we achieved a good prediction performance. Experiments on the MCL-JCI dataset showed a mean Bhattacharyya distance between the predicted and the original JND distributions of only 0.072.

[1]  Qin Huang,et al.  Prediction of Satisfied User Ratio for Compressed Video , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Qin Huang,et al.  Measure and Prediction of HEVC Perceptually Lossy/Lossless Boundary QP Values , 2017, 2017 Data Compression Conference (DCC).

[4]  Mohamed Cheriet,et al.  Mean Deviation Similarity Index: Efficient and Reliable Full-Reference Image Quality Evaluator , 2016, IEEE Access.

[5]  Xin Jin,et al.  VideoSet: A large-scale compressed video quality dataset based on JND measurement , 2017, J. Vis. Commun. Image Represent..

[6]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Dietmar Saupe,et al.  Disregarding the Big Picture: Towards Local Image Quality Assessment , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[8]  Yun Zhang,et al.  Interactive Subjective Study on Picture-level Just Noticeable Difference of Compressed Stereoscopic Images , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Chao Yang,et al.  Analysis and Prediction of JND-Based Video Quality Model , 2018, 2018 Picture Coding Symposium (PCS).

[11]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[12]  C.-C. Jay Kuo,et al.  Statistical Study on Perceived JPEG Image Quality via MCL-JCI Dataset Construction and Analysis , 2016, IQSP.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[15]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[16]  Ping Wang,et al.  MCL-JCV: A JND-based H.264/AVC video quality assessment dataset , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[17]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.