Identification of deep network generated images using disparities in color components

Abstract With the powerful deep network architectures, such as generative adversarial networks, one can easily generate photorealistic images. Although the generated images are not dedicated for fooling human or deceiving biometric authentication systems, research communities and public media have shown great concerns on the security issues caused by these images. This paper addresses the problem of identifying deep network generated (DNG) images. Taking the differences between camera imaging and DNG image generation into considerations, we analyze the disparities between DNG images and real images in different color components. We observe that the DNG images are more distinguishable from real ones in the chrominance components, especially in the residual domain. Based on these observations, we propose a feature set to capture color image statistics for identifying DNG images. Additionally, we evaluate several detection situations, including the training-testing data are matched or mismatched in image sources or generative models and detection with only real images. Extensive experimental results show that the proposed method can accurately identify DNG images and outperforms existing methods when the training and testing data are mismatched. Moreover, when the GAN model is unknown, our methods also achieves good performance with one-class classification by using only real images for training.

[1]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[2]  Jiwu Huang,et al.  Identification of Various Image Operations Using Residual-Based Features , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[4]  Jukka Komulainen,et al.  Face Spoofing Detection Using Colour Texture Analysis , 2016, IEEE Transactions on Information Forensics and Security.

[5]  Jiwu Huang,et al.  Image Forgery Localization via Integrating Tampering Possibility Maps , 2017, IEEE Transactions on Information Forensics and Security.

[6]  Honglak Lee,et al.  Learning to Align from Scratch , 2012, NIPS.

[7]  Tieniu Tan,et al.  Live face detection based on the analysis of Fourier spectra , 2004, SPIE Defense + Commercial Sensing.

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Koray Kavukcuoglu,et al.  Pixel Recurrent Neural Networks , 2016, ICML.

[10]  Xin Yang,et al.  Exposing Deep Fakes Using Inconsistent Head Poses , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Davide Cozzolino,et al.  Detection of GAN-Generated Fake Images over Social Networks , 2018, 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR).

[12]  Anil K. Jain,et al.  Face Spoof Detection With Image Distortion Analysis , 2015, IEEE Transactions on Information Forensics and Security.

[13]  Xin Yang,et al.  Exposing GAN-synthesized Faces Using Landmark Locations , 2019, IH&MMSec.

[14]  Alessandro Piva,et al.  Image Forgery Localization via Block-Grained Analysis of JPEG Artifacts , 2012, IEEE Transactions on Information Forensics and Security.

[15]  Wei Wei,et al.  COCO-GAN: Generation by Parts via Conditional Coordinating , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[17]  Siwei Lyu,et al.  Exposing DeepFake Videos By Detecting Face Warping Artifacts , 2018, CVPR Workshops.

[18]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[20]  B. S. Manjunath,et al.  Detecting GAN generated Fake Images using Co-occurrence Matrices , 2019, Media Watermarking, Security, and Forensics.

[21]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[22]  Ole Winther,et al.  Autoencoding beyond pixels using a learned similarity metric , 2015, ICML.

[23]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[24]  Jessica J. Fridrich,et al.  Rich Models for Steganalysis of Digital Images , 2012, IEEE Transactions on Information Forensics and Security.

[25]  Xu Zhang,et al.  Detecting and Simulating Artifacts in GAN Fake Images , 2019, 2019 IEEE International Workshop on Information Forensics and Security (WIFS).

[26]  Junichi Yamagishi,et al.  MesoNet: a Compact Facial Video Forgery Detection Network , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[27]  Sébastien Marcel,et al.  Biometric Antispoofing Methods: A Survey in Face Recognition , 2014, IEEE Access.

[28]  Jiwu Huang,et al.  Multi-Scale Fusion for Improved Localization of Malicious Tampering in Digital Images , 2016, IEEE Transactions on Image Processing.

[29]  Yann LeCun,et al.  Energy-based Generative Adversarial Networks , 2016, ICLR.

[30]  Yinda Zhang,et al.  LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop , 2015, ArXiv.

[31]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[33]  Luisa Verdoliva,et al.  Do GANs Leave Artificial Fingerprints? , 2018, 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR).

[34]  Zhaopeng Xu,et al.  On the robustness of JPEG post-compression to resampling factor estimation , 2020, Signal Process..

[35]  Yao Zhao,et al.  Detection of operation chain: JPEG-Resampling-JPEG , 2017, Signal Process. Image Commun..

[36]  Belhassen Bayar,et al.  Constrained Convolutional Neural Networks: A New Approach Towards General Purpose Image Manipulation Detection , 2018, IEEE Transactions on Information Forensics and Security.

[37]  Jiwu Huang,et al.  JPEG Error Analysis and Its Applications to Digital Image Forensics , 2010, IEEE Transactions on Information Forensics and Security.

[38]  Pawel Korus,et al.  Digital image integrity - a survey of protection and verification techniques , 2017, Digit. Signal Process..

[39]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[40]  Sébastien Marcel,et al.  Face Anti-spoofing Based on General Image Quality Assessment , 2014, 2014 22nd International Conference on Pattern Recognition.

[41]  Hyeonjoon Moon,et al.  Deep Learning Based Computer Generated Face Identification Using Convolutional Neural Network , 2018, Applied Sciences.

[42]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[43]  Christian Riess,et al.  Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations , 2019, 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[44]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[45]  Andreas Rössler,et al.  FaceForensics++: Learning to Detect Manipulated Facial Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  LinLin Shen,et al.  Deep Feature Consistent Variational Autoencoder , 2016, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[47]  Alex Graves,et al.  Conditional Image Generation with PixelCNN Decoders , 2016, NIPS.

[48]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  K. J. Ray Liu,et al.  Forensically determining the order of signal processing operations , 2013, 2013 IEEE International Workshop on Information Forensics and Security (WIFS).

[50]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[51]  Cristiano Saltori,et al.  Incremental learning for the detection and classification of GAN-generated images , 2019, 2019 IEEE International Workshop on Information Forensics and Security (WIFS).

[52]  Siwei Lyu,et al.  In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[53]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[54]  Chih-Chung Hsu,et al.  Detecting Generated Image Based on a Coupled Network with Two-Step Pairwise Learning , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[55]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[56]  Scott McCloskey,et al.  Detecting GAN-Generated Imagery Using Saturation Cues , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[57]  Hao Li,et al.  Protecting World Leaders Against Deep Fakes , 2019, CVPR Workshops.

[58]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[59]  Bolin Chen,et al.  Fake Faces Identification via Convolutional Neural Network , 2018, IH&MMSec.

[60]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[61]  Andreas Rössler,et al.  ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection , 2018, ArXiv.

[62]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[63]  Anil K. Jain,et al.  Secure Face Unlock: Spoof Detection on Smartphones , 2016, IEEE Transactions on Information Forensics and Security.

[64]  Jessica J. Fridrich,et al.  Ensemble Classifiers for Steganalysis of Digital Media , 2012, IEEE Transactions on Information Forensics and Security.

[65]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[66]  Hongwei Liu,et al.  Variational probabilistic generative framework for single image super-resolution , 2019, Signal Process..