We Need No Pixels: Video Manipulation Detection Using Stream Descriptors

Manipulating video content is easier than ever. Due to the misuse potential of manipulated content, multiple detection techniques that analyze the pixel data from the videos have been proposed. However, clever manipulators should also carefully forge the metadata and auxiliary header information, which is harder to do for videos than images. In this paper, we propose to identify forged videos by analyzing their multimedia stream descriptors with simple binary classifiers, completely avoiding the pixel space. Using well-known datasets, our results show that this scalable approach can achieve a high manipulation detection score if the manipulators have not done a careful data sanitization of the multimedia stream descriptors.

[1]  Lucas Theis,et al.  Fast Face-Swap Using Convolutional Neural Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Jiwu Huang,et al.  Image Forgery Localization via Integrating Tampering Possibility Maps , 2017, IEEE Transactions on Information Forensics and Security.

[3]  Keith Jack.  Chapter 13 – MPEG-2 , 2007 .

[4]  Robert Chesney,et al.  Disinformation on Steroids: The Threat of Deep Fakes , 2018 .

[5]  K. J. Ray Liu,et al.  Forensics vs. anti-forensics: A decision and game theoretic framework , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Christian Riess,et al.  Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations , 2019, 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[7]  Paolo Bestagini,et al.  NEAR-DUPLICATE VIDEO DETECTION EXPLOITING NOISE RESIDUAL TRACES , 2017 .

[8]  Paolo Bestagini,et al.  Codec and GOP Identification in Double Compressed Videos , 2016, IEEE Transactions on Image Processing.

[9]  Wes McKinney,et al.  Data Structures for Statistical Computing in Python , 2010, SciPy.

[10]  Nasir D. Memon,et al.  Video copy detection based on source device characteristics: a complementary approach to content-based methods , 2008, MIR '08.

[11]  Hyrum S. Anderson,et al.  The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation , 2018, ArXiv.

[12]  Jonathan G. Fiscus,et al.  MFC Datasets: Large-Scale Benchmark Datasets for Media Forensic Challenge Evaluation , 2019, 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[13]  Chia-Wen Lin,et al.  Video forgery detection using correlation of noise residue , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[14]  Sébastien Marcel,et al.  DeepFakes: a New Threat to Face Recognition? Assessment and Detection , 2018, ArXiv.

[15]  Jan P. Allebach,et al.  Forensic techniques for classifying scanner, computer generated and digital camera images , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Paolo Bestagini,et al.  Blind Detection and Localization of Video Temporal Splicing Exploiting Sensor-Based Footprints , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[17]  Andreas Rössler,et al.  FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces , 2018, ArXiv.

[18]  Justus Thies,et al.  Supplemental Material for ” Face 2 Face : Real-time Face Capture and Reenactment of RGB Videos ” , 2016 .

[19]  Jiwu Huang,et al.  Exposing Fake Bit Rate Videos and Estimating Original Bit Rates , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Justus Thies,et al.  Face2Face: Real-Time Face Capture and Reenactment of RGB Videos , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Siwei Lyu,et al.  In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[22]  Davide Cozzolino,et al.  Residual-based forensic comparison of video sequences , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[23]  Orhan Bulan,et al.  Geometric distortion signatures for printer identification , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[24]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[25]  Takaya Saito,et al.  The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets , 2015, PloS one.

[26]  Davide Cozzolino,et al.  Autoencoder with recurrent neural networks for video forgery detection , 2017, Media Watermarking, Security, and Forensics.

[27]  Davide Cozzolino,et al.  A PatchMatch-Based Dense-Field Algorithm for Video Copy–Move Detection and Localization , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Marco Fontani,et al.  A Video Forensic Framework for the Unsupervised Analysis of MP4-Like File Container , 2019, IEEE Transactions on Information Forensics and Security.

[29]  Alessandro Piva,et al.  Image Forgery Localization via Block-Grained Analysis of JPEG Artifacts , 2012, IEEE Transactions on Information Forensics and Security.

[30]  K. J. Ray Liu,et al.  Temporal Forensics and Anti-Forensics for Motion Compensated Video , 2012, IEEE Transactions on Information Forensics and Security.

[31]  Alex ChiChung Kot,et al.  Modeling the EXIF-Image correlation for image manipulation detection , 2011, 2011 18th IEEE International Conference on Image Processing.

[32]  Paolo Bestagini,et al.  Local tampering detection in video sequences , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[33]  Edward J. Delp,et al.  Deepfake Video Detection Using Recurrent Neural Networks , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[34]  A. Piva,et al.  overview paper An overview on video forensics , 2012 .

[35]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[36]  Yaser Sheikh,et al.  Recycle-GAN: Unsupervised Video Retargeting , 2018, ECCV.

[37]  Andrew Owens,et al.  Fighting Fake News: Image Splice Detection via Learned Self-Consistency , 2018, ECCV.

[38]  Savvas Zannettou,et al.  The Good, the Bad and the Bait: Detecting and Characterizing Clickbait on YouTube , 2018, 2018 IEEE Security and Privacy Workshops (SPW).