FOCAL: A Forgery Localization Framework based on Video Coding Self-Consistency

Forgery operations on video contents are nowadays within the reach of anyone, thanks to the availability of powerful and user-friendly editing software. Integrity verification and authentication of videos represent a major interest in both journalism (e.g., fake news debunking) and legal environments dealing with digital evidence (e.g., a court of law). While several strategies and different forensics traces have been proposed in recent years, latest solutions aim at increasing the accuracy by combining multiple detectors and features. This paper presents a video forgery localization framework that verifies the self-consistency of coding traces between and within video frames, by fusing the information derived from a set of independent feature descriptors. The feature extraction step is carried out by means of an explainable convolutional neural network architecture, specifically designed to look for and classify coding artifacts. The overall framework was validated in two typical forgery scenarios: temporal and spatial splicing. Experimental results show an improvement to the state-of-the-art on temporal splicing localization and also promising performance in the newly tackled case of spatial splicing, on both synthetic and real-world videos.

[1]  Jiwu Huang,et al.  Exposing Fake Bit Rate Videos and Estimating Original Bit Rates , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Nasir D. Memon,et al.  Video copy detection based on source device characteristics: a complementary approach to content-based methods , 2008, MIR '08.

[3]  Paolo Bestagini,et al.  Video Codec Forensics Based on Convolutional Neural Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[4]  Qingzhong Liu,et al.  Detection of Double MPEG-2 Compression Based on Distributions of DCT coefficients , 2013, Int. J. Pattern Recognit. Artif. Intell..

[5]  David Vazquez-Padin,et al.  Detection of video double encoding with GOP size estimation , 2012, 2012 IEEE International Workshop on Information Forensics and Security (WIFS).

[6]  Mo Chen,et al.  Source digital camcorder identification using sensor photo response non-uniformity , 2007, Electronic Imaging.

[7]  Weihong Wang,et al.  Exposing digital forgeries in video by detecting double MPEG compression , 2006, MM&Sec '06.

[8]  Davide Cozzolino,et al.  Autoencoder with recurrent neural networks for video forgery detection , 2017, Media Watermarking, Security, and Forensics.

[9]  Raahat Devender Singh,et al.  Inter-frame forgery detection in H.264 videos using motion and brightness gradients , 2017, Multimedia Tools and Applications.

[10]  K. J. Ray Liu,et al.  Temporal Forensics and Anti-Forensics for Motion Compensated Video , 2012, IEEE Transactions on Information Forensics and Security.

[11]  Yun Q. Shi,et al.  Detection of Double Compression in MPEG-4 Videos Based on Markov Statistics , 2013, IEEE Signal Processing Letters.

[12]  Alberto Del Bimbo,et al.  Ieee Transactions on Information Forensics and Security 1 a Sift-based Forensic Method for Copy-move Attack Detection and Transformation Recovery , 2022 .

[13]  Sanjay Kumar Singh,et al.  Passive copy-move forgery detection in videos , 2014, 2014 International Conference on Computer and Communication Technology (ICCCT).

[14]  Pier Luigi Dragotti,et al.  Video jitter analysis for automatic bootleg detection , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  K. Sitara,et al.  Digital video tampering detection: An overview of passive techniques , 2016, Digit. Investig..

[17]  Mauro Barni,et al.  Detection of malevolent changes in digital video for forensic applications , 2007, Electronic Imaging.

[18]  Wei Zhang,et al.  Detecting Removed Object from Video with Stationary Background , 2012, IWDW.

[19]  Tianqiang Huang,et al.  Using similarity analysis to detect frame duplication forgery in videos , 2014, Multimedia Tools and Applications.

[20]  Mark D. Ryan Ben Smyth GuilinWang.  Information Security Practice and Experience , 2012, Lecture Notes in Computer Science.

[21]  Paolo Bestagini,et al.  Codec and GOP Identification in Double Compressed Videos , 2016, IEEE Transactions on Image Processing.

[22]  Sabu Emmanuel,et al.  Video forgery detection and localization using normalized cross-correlation of moment features , 2016, 2016 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI).

[23]  Weihong Wang,et al.  Exposing Digital Forgeries in Interlaced and Deinterlaced Video , 2007, IEEE Transactions on Information Forensics and Security.

[24]  Zhenzhen Zhang,et al.  Efficient video frame insertion and deletion detection based on inconsistency of correlations between local binary pattern coded frames , 2015, Secur. Commun. Networks.

[25]  Sabu Emmanuel,et al.  Video forgery detection using HOG features and compression properties , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[26]  Paolo Bestagini,et al.  Video codec identification , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Jie Liu,et al.  Exposing Digital Video Forgery by Detecting Motion-Compensated Edge Artifact , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[28]  Siwei Lyu,et al.  In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[29]  Paolo Bestagini,et al.  Local tampering detection in video sequences , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[30]  Paolo Bestagini,et al.  An overview on video forensics , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[31]  Yun Q. Shi,et al.  Inter-frame Video Forgery Detection Based on Block-Wise Brightness Variance Descriptor , 2014, IWDW.

[32]  Takahiro Okabe,et al.  Detecting Forgery From Static-Scene Video Based on Inconsistency in Noise Level Functions , 2010, IEEE Transactions on Information Forensics and Security.

[33]  Hongmei Liu,et al.  Detecting Frame Deletion in H.264 Video , 2014, ISPEC.

[34]  Shilin Wang,et al.  Identifying Video Forgery Process Using Optical Flow , 2013, IWDW.

[35]  Bin Li,et al.  Automatic Detection of Object-Based Forgery in Advanced Video , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Davide Cozzolino,et al.  Splicebuster: A new blind image splicing detector , 2015, 2015 IEEE International Workshop on Information Forensics and Security (WIFS).

[37]  Paolo Bestagini,et al.  Multiple compression detection for video sequences , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[38]  Paolo Bestagini,et al.  Detection of temporal interpolation in video sequences , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[39]  Jiwu Huang,et al.  Detection of double compression with the same bit rate in MPEG-2 videos , 2014, 2014 IEEE China Summit & International Conference on Signal and Information Processing (ChinaSIP).

[40]  Heung-Kyu Lee,et al.  Detection of Upscale-Crop and Partial Manipulation in Surveillance Video Based on Sensor Pattern Noise , 2013, Sensors.

[41]  Edward J. Delp,et al.  Deepfake Video Detection Using Recurrent Neural Networks , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[42]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[43]  Davide Cozzolino,et al.  Video forgery detection and localization based on 3D patchmatch , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[44]  Andrew Owens,et al.  Fighting Fake News: Image Splice Detection via Learned Self-Consistency , 2018, ECCV.

[45]  James F. O'Brien,et al.  Exposing Digital Forgeries in Ballistic Motion , 2012, IEEE Transactions on Information Forensics and Security.

[46]  Gaurav Gupta,et al.  Compression noise based video forgery detection , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[47]  Guo-Shiang Lin,et al.  Detecting frame duplication based on spatial and temporal analyses , 2011, 2011 6th International Conference on Computer Science & Education (ICCSE).

[48]  Shahrokh Ghaemmaghami,et al.  Detection of regional copy/move forgery in MPEG videos using optical flow , 2015, 2015 The International Symposium on Artificial Intelligence and Signal Processing (AISP).