Does deep machine vision have just noticeable difference (JND)?

As an important perceptual characteristic of the Human Visual System (HVS), the Just Noticeable Difference (JND) has been studied for decades with image/video processing (e.g., perceptual image/video coding). However, there is little exploration on the existence of JND for AI, like Deep Machine Vision (DMV), although the DMV has made great strides in many machine vision tasks. In this paper, we take an initial attempt, and demonstrate that DMV does have the JND, termed as DMVJND. Besides, we propose a JND model for the classification task in DMV. It has been discovered that DMV can tolerate distorted images with average PSNR of only 9.56dB (the lower the better), by generating JND via unsupervised learning with our DMVJND-NET. In particular, a semantic-guided redundancy assessment strategy is designed to constrain the magnitude and spatial distribution of the JND. Experimental results on classification tasks demonstrate that we successfully find and model the JND for deep machine vision. Meanwhile, our DMV-JND paves a possible direction for DMV oriented image/video compression, watermarking, quality assessment, deep neural network security, and so on.

[1]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yi Yang,et al.  Self-produced Guidance for Weakly-supervised Object Localization , 2018, ECCV.

[3]  Kuo-Cheng Liu,et al.  A Perceptually Tuned Watermarking Scheme for Color Images , 2010, IEEE Transactions on Image Processing.

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Bin Fang,et al.  Just Noticeable Distortion-Based Perceptual Rate Control in HEVC , 2020, IEEE Transactions on Image Processing.

[6]  Sung-Ho Bae,et al.  An HEVC-Compliant Perceptual Video Coding Scheme Based on JND Models for Variable Block-Sized Transform Kernels , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Tuyet-Trang Lam,et al.  Selective Error Detection for Error-Resilient Wavelet-Based Image Coding , 2007, IEEE Transactions on Image Processing.

[8]  Jun Zhu,et al.  Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Susu Yao,et al.  Just noticeable distortion model and its applications in video coding , 2005, Signal Process. Image Commun..

[10]  Xin Jin,et al.  VideoSet: A large-scale compressed video quality dataset based on JND measurement , 2017, J. Vis. Commun. Image Represent..

[11]  Guangming Shi,et al.  Enhanced Just Noticeable Difference Model for Images With Pattern Complexity , 2017, IEEE Transactions on Image Processing.

[12]  Yu-Bin Yang,et al.  Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections , 2016, ArXiv.

[13]  Nam Ling,et al.  H.264/Advanced Video Control Perceptual Optimization Coding Based on JND-Directed Coefficient Suppression , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Zhe L. Lin,et al.  Top-Down Neural Attention by Excitation Backprop , 2016, International Journal of Computer Vision.

[15]  Jiro Katto,et al.  Performance Comparison of Convolutional AutoEncoders, Generative Adversarial Networks and Super-Resolution for Image Compression , 2018, CVPR Workshops.

[16]  Jan Kautz,et al.  Video-to-Video Synthesis , 2018, NeurIPS.

[17]  Munchurl Kim,et al.  A Novel DCT-Based JND Model for Luminance Adaptation Effect in DCT Frequency , 2013, IEEE Signal Processing Letters.

[18]  Seyed-Mohsen Moosavi-Dezfooli,et al.  Universal Adversarial Perturbations , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[20]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[21]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Yiwen Guo,et al.  Subspace Attack: Exploiting Promising Subspaces for Query-Efficient Black-box Attacks , 2019, NeurIPS.

[24]  Kai Wang,et al.  Just Noticeable Distortion Profile for Flat-Shaded 3D Mesh Surfaces , 2016, IEEE Transactions on Visualization and Computer Graphics.

[25]  Timo Aila,et al.  Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder , 2017, ACM Trans. Graph..

[26]  Seyed-Mohsen Moosavi-Dezfooli,et al.  DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Samy Bengio,et al.  Adversarial Machine Learning at Scale , 2016, ICLR.

[28]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  C.-C. Jay Kuo,et al.  Statistical Study on Perceived JPEG Image Quality via MCL-JCI Dataset Construction and Analysis , 2016, IQSP.

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Yonggang Wen,et al.  Deepqoe: A Unified Framework for Learning to Predict Video QoE , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[32]  King Ngi Ngan,et al.  Spatio-Temporal Just Noticeable Distortion Profile for Grey Scale Image/Video in DCT Domain , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Chun-Hsien Chou,et al.  A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile , 1995, IEEE Trans. Circuits Syst. Video Technol..

[34]  Zhuo Chen,et al.  Toward Intelligent Sensing: Intermediate Deep Feature Compression , 2020, IEEE Transactions on Image Processing.

[35]  Zoran A. Ivanovski,et al.  An Efficient Selective Perceptual-Based Super-Resolution Estimator , 2011, IEEE Transactions on Image Processing.

[36]  Yao Zhao,et al.  Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Aleksander Madry,et al.  Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors , 2018, ICLR.

[39]  Robert J. Safranek,et al.  Signal compression based on models of human perception , 1993, Proc. IEEE.

[40]  Yi Yang,et al.  Adversarial Complementary Learning for Weakly Supervised Object Localization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Manoranjan Paul,et al.  Just Noticeable Difference for Images With Decomposition Model for Separating Edge and Textured Regions , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  C.-C. Jay Kuo,et al.  Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression , 2020, IEEE Transactions on Image Processing.

[43]  Jun Zhu,et al.  Improving Black-box Adversarial Attacks with a Transfer-based Prior , 2019, NeurIPS.