Exposing DeepFake Videos By Detecting Face Warping Artifacts

In this work, we describe a new deep learning based method that can effectively distinguish AI-generated fake videos (referred to as {\em DeepFake} videos hereafter) from real videos. Our method is based on the observations that current DeepFake algorithm can only generate images of limited resolutions, which need to be further warped to match the original faces in the source video. Such transforms leave distinctive artifacts in the resulting DeepFake videos, and we show that they can be effectively captured by convolutional neural networks (CNNs). Compared to previous methods which use a large amount of real and DeepFake generated images to train CNN classifier, our method does not need DeepFake generated images as negative training examples since we target the artifacts in affine face warping as the distinctive feature to distinguish real and fake images. The advantages of our method are two-fold: (1) Such artifacts can be simulated directly using simple image processing operations on a image to make it as negative example. Since training a DeepFake model to generate negative examples is time-consuming and resource-demanding, our method saves a plenty of time and resources in training data collection; (2) Since such artifacts are general existed in DeepFake videos from different sources, our method is more robust compared to others. Our method is evaluated on two sets of DeepFake video datasets for its effectiveness in practice.

[1]  Hany Farid,et al.  Exposing digital forgeries by detecting traces of resampling , 2005, IEEE Transactions on Signal Processing.

[2]  S. Prasad,et al.  On Resampling Detection and its Application to Detect Image Tampering , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[3]  Matthias Kirchner,et al.  Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue , 2008, MM&Sec '08.

[4]  Babak Mahdian,et al.  Ieee Transactions on Information Forensics and Security 1 Blind Authentication Using Periodic Properties of Interpolation , 2022 .

[5]  Rainer Böhme,et al.  Hiding Traces of Resampling in Digital Images , 2008, IEEE Transactions on Information Forensics and Security.

[6]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[7]  Brian C. Lovell,et al.  Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference , 2009, ICB.

[8]  Matthias Kirchner,et al.  On resampling detection in re-compressed images , 2009, 2009 First IEEE International Workshop on Information Forensics and Security (WIFS).

[9]  Fernando Pérez-González,et al.  On the role of differentiation for resampling detection , 2010, 2010 IEEE International Conference on Image Processing.

[10]  Nenghai Yu,et al.  Image Forensics with Rotation-Tolerant Resampling Detection , 2012, 2012 IEEE International Conference on Multimedia and Expo Workshops.

[11]  Stefan Katzenbeisser,et al.  Robust Resampling Detection in Digital Images , 2012, Communications and Multimedia Security.

[12]  Gang Xiong,et al.  Image resampling detection based on texture classification , 2013, Multimedia Tools and Applications.

[13]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[14]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[15]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[18]  Lior Wolf,et al.  Unsupervised Cross-Domain Image Generation , 2016, ICLR.

[19]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[20]  Larry S. Davis,et al.  Two-Stream Neural Networks for Tampered Face Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Tomas Pfister,et al.  Learning from Simulated and Unsupervised Images through Adversarial Training , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[24]  B. S. Manjunath,et al.  Detection and Localization of Image Forgeries Using Resampling Features and Deep Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[26]  Scott McCloskey,et al.  Detecting GAN-generated Imagery using Color Cues , 2018, ArXiv.

[27]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Edward J. Delp,et al.  Deepfake Video Detection Using Recurrent Neural Networks , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[29]  Junichi Yamagishi,et al.  MesoNet: a Compact Facial Video Forgery Detection Network , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[30]  Siwei Lyu,et al.  In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking , 2018, ArXiv.

[31]  Sébastien Marcel,et al.  DeepFakes: a New Threat to Face Recognition? Assessment and Detection , 2018, ArXiv.

[32]  Bin Li,et al.  Detection of Deep Network Generated Images Using Disparities in Color Components , 2018, ArXiv.

[33]  Yaser Sheikh,et al.  Recycle-GAN: Unsupervised Video Retargeting , 2018, ECCV.

[34]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[35]  Justus Thies,et al.  Face2Face: real-time face capture and reenactment of RGB videos , 2019, Commun. ACM.

[36]  Xin Yang,et al.  Exposing Deep Fakes Using Inconsistent Head Poses , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Davide Cozzolino,et al.  Noiseprint: A CNN-Based Camera Model Fingerprint , 2018, IEEE Transactions on Information Forensics and Security.

[38]  Bin Li,et al.  Identification of deep network generated images using disparities in color components , 2020, Signal Process..