Extracting camera-based fingerprints for video forensics

Video source attribution is an important operation in forensics applications. Identifying which specific device or camera model took a video can help in authorship verification, but can be also a precious source of information for detecting a possible manipulation. The key observation is that any physical device leaves peculiar traces in the acquired content, a sort of fingerprint that can be exploited to establish data provenance. Moreover, absence or modification of such traces may reveal a possible manipulation. In this paper, inspired by recent work on images, we train a neural network that enhances the model-related traces hidden in a video, extracting a sort of camera fingerprint, called video noiseprint. The net is trained on pristine videos with a Siamese strategy, minimizing distances between same-model patches, and maximizing distances between unrelated patches. Experiments show that methods based on video noiseprints perform well in major forensic tasks, such as camera model identification and video forgery localization, with no need of prior knowledge on the specific manipulation or any form of fine-tuning.

[1]  A. Piva,et al.  overview paper An overview on video forensics , 2012 .

[2]  Antoine Doucet,et al.  Find it! Fraud Detection Contest Report , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[3]  Mo Chen,et al.  Source digital camcorder identification using sensor photo response non-uniformity , 2007, Electronic Imaging.

[4]  Andrew Owens,et al.  Fighting Fake News: Image Splice Detection via Learned Self-Consistency , 2018, ECCV.

[5]  Xin Yang,et al.  Exposing Deep Fakes Using Inconsistent Head Poses , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Christian Riess,et al.  Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations , 2019, 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[7]  Paolo Bestagini,et al.  Facing Device Attribution Problem for Stabilized Video Sequences , 2018, IEEE Transactions on Information Forensics and Security.

[8]  Siwei Lyu,et al.  In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[9]  Marco Fontani,et al.  VISION: a video and image dataset for source identification , 2017, EURASIP Journal on Information Security.

[10]  Marco Fontani,et al.  Hybrid reference-based Video Source Identification , 2019, Sensors.

[11]  Jean-Luc Dugelay,et al.  Videos versus still images: Asymmetric sensor pattern noise comparison on mobile phones , 2017, Media Watermarking, Security, and Forensics.

[12]  Andreas Rössler,et al.  FaceForensics++: Learning to Detect Manipulated Facial Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Nasir D. Memon,et al.  Source camera attribution using stabilized video , 2016, 2016 IEEE International Workshop on Information Forensics and Security (WIFS).

[14]  Paolo Bestagini,et al.  Local tampering detection in video sequences , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[15]  Mauro Barni,et al.  A video forensic technique for detecting frame deletion and insertion , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Ahmet Emir Dirik,et al.  PRNU-based source device attribution for YouTube videos , 2019, Digit. Investig..

[17]  David Vazquez-Padin,et al.  Localization of forgeries in MPEG-2 video through GOP size and DQ analysis , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[18]  Mo Chen,et al.  Determining Image Origin and Integrity Using Sensor Noise , 2008, IEEE Transactions on Information Forensics and Security.

[19]  Xing Zhang,et al.  Exposing Region Splicing Forgeries with Blind Local Noise Estimation , 2013, International Journal of Computer Vision.

[20]  Davide Cozzolino,et al.  Autoencoder with recurrent neural networks for video forgery detection , 2017, Media Watermarking, Security, and Forensics.

[21]  Mauro Barni,et al.  Detection of malevolent changes in digital video for forensic applications , 2007, Electronic Imaging.

[22]  Matthew C. Stamm,et al.  Learned Forensic Source Similarity for Unknown Camera Models , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Min Wu,et al.  Exploring compression effects for improved source camera identification using strongly compressed video , 2011, 2011 18th IEEE International Conference on Image Processing.

[24]  Edward J. Delp,et al.  Deepfake Video Detection Using Recurrent Neural Networks , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[25]  Weihong Wang,et al.  Exposing digital forgeries in video by detecting double MPEG compression , 2006, MM&Sec '06.

[26]  Davide Cozzolino,et al.  Noiseprint: A CNN-Based Camera Model Fingerprint , 2018, IEEE Transactions on Information Forensics and Security.

[27]  Chia-Wen Lin,et al.  Video forgery detection using correlation of noise residue , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[28]  Zhengquan Xu,et al.  Automatic location of frame deletion point for digital video forensics , 2014, IH&MMSec '14.

[29]  Andreas Rössler,et al.  ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection , 2018, ArXiv.

[30]  Davide Cozzolino,et al.  A PatchMatch-Based Dense-Field Algorithm for Video Copy–Move Detection and Localization , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Junichi Yamagishi,et al.  MesoNet: a Compact Facial Video Forgery Detection Network , 2018, 2018 IEEE International Workshop on Information Forensics and Security (WIFS).

[32]  Jessica J. Fridrich,et al.  Large scale test of sensor fingerprint camera identification , 2009, Electronic Imaging.

[33]  Davide Cozzolino,et al.  Splicebuster: A new blind image splicing detector , 2015, 2015 IEEE International Workshop on Information Forensics and Security (WIFS).

[34]  Miroslav Goljan,et al.  Digital camera identification from sensor pattern noise , 2006, IEEE Transactions on Information Forensics and Security.

[35]  Luisa Verdoliva,et al.  A Bayesian-MRF Approach for PRNU-Based Image Forgery Detection , 2014, IEEE Transactions on Information Forensics and Security.