Local Pixel Manipulation Detection with Deep Neural Networks

In times of digital images and easy to achieve manipulations, tampered photos are ubiquitous. Well carried out manipulations are almost impossible to identify, even for humans. To earn back trust it is necessary to develop techniques that can detect such manipulations. Two of the most commonly used manipulations are splicing and copy-move, e.g., copy a part from one image and paste in into another or the same image. In this thesis, we present an approach based on deep learning that significantly improves the possibilities to detect such manipulations. The approach uses a combination of a VGG-Net for feature extraction and Global Average Pooling for classification as its architecture. Our approach not only classifies the images, for being manipulated or not, but also localizes where the manipulations occur. To achieve this localization, it uses patches combined with image classification. We evaluate our approach and conduct several experiments, in which we compare the rate of classification against other well-known techniques, such as SIFT (copy-move), Expectation Maximization with segmentation (splicing) and a retrained Faster-RCNN (both). Our approach achieves the highest F1 score across all evaluated techniques, reaching about 77% macro average for our generated COCO segmentation data set for splicing and 79% for the Columbia data set, respectively.