Optical Flow and Prediction Residual Based Hybrid Forensic System for Inter-Frame Tampering Detection

In the wake of widespread proliferation of inexpensive and easy-to-use digital content editing software, digital videos have lost the idealized reputation they once held as universal, objective and infallible evidence of occurrence of events. The pliability of digital content and its innate vulnerability to unobtrusive alterations causes us to become skeptical of its validity. However, in spite of the fact that digital videos may not always present a truthful picture of reality, their usefulness in today’s world is incontrovertible. Therefore, the need to verify the integrity and authenticity of the contents of a digital video becomes paramount, especially in critical scenarios such as defense planning and legal trials where reliance on untrustworthy evidence could have grievous ramifications. Inter-frame tampering, which involves insertion/removal/replication of sets of frames into/from/within a video sequence, is among the most un-convoluted and elusive video forgeries. In this paper, we propose a potent hybrid forensic system that detects inter-frame forgeries in compressed videos. The system encompasses two forensic techniques. The first is a novel optical flow analysis based frame-insertion and removal detection procedure, where we focus on the brightness gradient component of optical flow and detect irregularities caused therein by post-production frame-tampering. The second component is a prediction residual examination based scheme that expedites detection and localization of replicated frames in video sequences. Subjective and quantitative results of comprehensive tests on an elaborate dataset under diverse experimental set-ups substantiate the effectuality and robustness of the proposed system.