Robust Invisible Hyperlinks in Physical Photographs Based on 3D Rendering Attacks

In the era of multimedia and Internet, people are eager to obtain information from offline to online. Quick Response (QR) codes and digital watermarks help us access information quickly. However, QR codes look ugly and invisible watermarks can be easily broken in physical photographs. Therefore, this paper proposes a novel method to embed hyperlinks into natural images, making the hyperlinks invisible for human eyes but detectable for mobile devices. Our method is an end-to-end neural network with an encoder to hide information and a decoder to recover information. From original images to physical photographs, camera imaging process will introduce a series of distortion such as noise, blur, and light. To train a robust decoder against the physical distortion from the real world, a distortion network based on 3D rendering is inserted between the encoder and the decoder to simulate the camera imaging process. Besides, in order to maintain the visual attraction of the image with hyperlinks, we propose a loss function based on just noticeable difference (JND) to supervise the training of encoder. Experimental results show that our approach outperforms the previous method in both simulated and real situations.

[1]  Kai Liu,et al.  Spatial Image Steganography Based on Generative Adversarial Network , 2018, ArXiv.

[2]  Bin Li,et al.  Automatic Steganographic Distortion Learning Using a Generative Adversarial Network , 2017, IEEE Signal Processing Letters.

[3]  Lei Xu,et al.  SteganoGAN: High Capacity Image Steganography with GANs , 2019, ArXiv.

[4]  Jia Liu,et al.  Generative Steganography with Kerckhoffs' Principle based on Generative Adversarial Networks , 2017, ArXiv.

[5]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Samy Bengio,et al.  Adversarial examples in the physical world , 2016, ICLR.

[7]  Weisi Lin,et al.  Estimating Just-Noticeable Distortion for Video , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Evgeny Burnaev,et al.  Steganographic generative adversarial networks , 2017, International Conference on Machine Vision.

[9]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Guangming Shi,et al.  Just Noticeable Difference Estimation for Images With Free-Energy Principle , 2013, IEEE Transactions on Multimedia.

[12]  Ren Ng,et al.  StegaStamp: Invisible Hyperlinks in Physical Photographs , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Gang Wang,et al.  Connecting the Digital and Physical World: Improving the Robustness of Adversarial Attacks , 2019, AAAI.

[14]  Samuel W. Hasinoff,et al.  Photon, Poisson Noise , 2014, Computer Vision, A Reference Guide.

[15]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[16]  Susu Yao,et al.  Just noticeable distortion model and its applications in video coding , 2005, Signal Process. Image Commun..

[17]  Li Fei-Fei,et al.  HiDDeN: Hiding Data With Deep Networks , 2018, ECCV.

[18]  Richard Shin.  JPEG-resistant Adversarial Images , 2017 .

[19]  Jing Dong,et al.  Deep learning for steganalysis via convolutional neural networks , 2015, Electronic Imaging.

[20]  Yan Ke,et al.  Generative steganography with Kerckhoffs’ principle , 2018, Multimedia Tools and Applications.

[21]  Toshihiro Matsui,et al.  Image Transformation can make Neural Networks more robust against Adversarial Examples , 2019, ArXiv.

[22]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[23]  Atul Prakash,et al.  Robust Physical-World Attacks on Deep Learning Visual Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[25]  Duen Horng Chau,et al.  ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector , 2018, ECML/PKDD.

[26]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[27]  Jing Dong,et al.  SSGAN: Secure Steganography Based on Generative Adversarial Networks , 2017, PCM.

[28]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[29]  Thomas Brox,et al.  Generating Images with Perceptual Similarity Metrics based on Deep Networks , 2016, NIPS.

[30]  Logan Engstrom,et al.  Synthesizing Robust Adversarial Examples , 2017, ICML.

[31]  Bin Li,et al.  CNN-Based Adversarial Embedding for Image Steganography , 2019, IEEE Transactions on Information Forensics and Security.

[32]  Minqing Zhang,et al.  Recent Advances of Image Steganography With Generative Adversarial Networks , 2019, IEEE Access.

[33]  Dawn Song,et al.  Physical Adversarial Examples for Object Detectors , 2018, WOOT @ USENIX Security Symposium.

[34]  Bin Li,et al.  A Novel Image Steganography Method via Deep Convolutional Generative Adversarial Networks , 2018, IEEE Access.

[35]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[36]  Guangming Shi,et al.  Enhanced Just Noticeable Difference Model for Images With Pattern Complexity , 2017, IEEE Transactions on Image Processing.

[37]  Sanjeev J. Koppal,et al.  Lambertian Reflectance , 2020, Computer Vision, A Reference Guide.

[38]  Manoranjan Paul,et al.  Just Noticeable Difference for Images With Decomposition Model for Separating Edge and Textured Regions , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Guangtao Zhai,et al.  The Invisible QR Code , 2015, ACM Multimedia.

[40]  Dwijendra K. Ray-Chaudhuri,et al.  Binary mixture flow with free energy lattice Boltzmann methods , 2022, arXiv.org.