Countering Spoof: Towards Detecting Deepfake with Multidimensional Biological Signals

The deepfake technology is conveniently abused with the low technology threshold, which may bring the huge social security risks. As GAN-based synthesis technology is becoming stronger, various methods are difficult to classify the fake content effectively. However, although the fake content generated by GANs can deceive the human eyes, it ignores the biological signals hidden in the face video. In this paper, we proposed a novel video forensics method with multidimensional biological signals, which extracting the difference of the biological signal between real and fake videos from three dimensions. The experimental results show that our method achieves 98% accuracy on the main public dataset. Compared with other technologies, the proposed method only extracts fake video information and is not limited to a specific generation method, so it is not affected by synthetic methods and has good adaptability.

[1]  Nenghai Yu,et al.  Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Andreas Rössler,et al.  FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces , 2018, ArXiv.

[3]  Siwei Lyu,et al.  In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[4]  Yuhui Zheng,et al.  A Serial Image Copy-Move Forgery Localization Scheme With Source/Target Distinguishment , 2020, IEEE Transactions on Multimedia.

[5]  Siwei Lyu,et al.  Exposing DeepFake Videos By Detecting Face Warping Artifacts , 2018, CVPR Workshops.

[6]  Hao Li,et al.  Protecting World Leaders Against Deep Fakes , 2019, CVPR Workshops.

[7]  Junichi Yamagishi,et al.  Distinguishing computer graphics from natural images using convolution neural networks , 2017, 2017 IEEE Workshop on Information Forensics and Security (WIFS).

[8]  Ilke Demir,et al.  FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals , 2019, IEEE transactions on pattern analysis and machine intelligence.

[9]  Andreas Rössler,et al.  FaceForensics++: Learning to Detect Manipulated Facial Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Ji Hou,et al.  Deep Face Forgery Detection , 2020, ArXiv.

[11]  Raymond Chiong,et al.  Remote heart rate measurement using low-cost RGB face video: a technical literature review , 2018, Frontiers of Computer Science.

[12]  Paolo Bestagini,et al.  Video Face Manipulation Detection Through Ensemble of CNNs , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[13]  Felix Juefei-Xu,et al.  FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces , 2019, IJCAI.

[14]  Edward J. Delp,et al.  Deepfake Video Detection Using Recurrent Neural Networks , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[15]  Seung-Hwan Bae,et al.  Generative Adversarial Ensemble Learning for Face Forensics , 2020, IEEE Access.

[16]  Belhassen Bayar,et al.  A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer , 2016, IH&MMSec.

[17]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Premkumar Natarajan,et al.  Recurrent Convolutional Strategies for Face Manipulation Detection in Videos , 2019, CVPR Workshops.

[19]  Nenghai Yu,et al.  Multi-attentional Deepfake Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Junichi Yamagishi,et al.  MesoNet: a Compact Facial Video Forgery Detection Network , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[21]  Junichi Yamagishi,et al.  Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Conrad S. Tucker,et al.  Bounded Kalman filter method for motion-robust, non-contact heart rate estimation. , 2018, Biomedical optics express.

[23]  Giulia Boato,et al.  Physiologically-based detection of computer generated faces in video , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[24]  Sébastien Marcel,et al.  DeepFakes: a New Threat to Face Recognition? Assessment and Detection , 2018, ArXiv.

[25]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Baining Guo,et al.  Face X-Ray for More General Face Forgery Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Xin Yang,et al.  Exposing Deep Fakes Using Inconsistent Head Poses , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Yuhui Zheng,et al.  Image splicing localization using residual image and residual-based fully convolutional network , 2020, J. Vis. Commun. Image Represent..

[29]  Nicu Sebe,et al.  Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Xilin Chen,et al.  RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation , 2019, IEEE Transactions on Image Processing.