Image Manipulation Detection by Multi-View Multi-Scale Supervision

The key challenge of image manipulation detection is how to learn generalizable features that are sensitive to manipulations in novel data, whilst specific to prevent false alarms on authentic images. Current research emphasizes the sensitivity, with the specificity overlooked. In this paper we address both aspects by multi-view feature learning and multi-scale supervision. By exploiting noise distribution and boundary artifact surrounding tampered regions, the former aims to learn semantic-agnostic and thus more generalizable features. The latter allows us to learn from authentic images which are nontrivial to be taken into account by current semantic segmentation network based methods. Our thoughts are realized by a new network which we term MVSS-Net. Extensive experiments on five benchmark sets justify the viability of MVSS-Net for both pixellevel and image-level manipulation detection.

[1]  Larry S. Davis,et al.  Learning Rich Features for Image Manipulation Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Jessica J. Fridrich,et al.  Rich Models for Steganalysis of Digital Images , 2012, IEEE Transactions on Information Forensics and Security.

[3]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jing Dong,et al.  CASIA Image Tampering Detection Evaluation Database , 2013, 2013 IEEE China Summit and International Conference on Signal and Information Processing.

[5]  Lior Wolf,et al.  Wish You Were Here: Context-Aware Human Generation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Larry S. Davis,et al.  Generate, Segment, and Refine: Towards Generic Manipulation Segmentation , 2018, AAAI.

[7]  Bin Jiang,et al.  Constrained R-Cnn: A General Image Manipulation Detection Model , 2020, 2020 IEEE International Conference on Multimedia and Expo (ICME).

[8]  Gang Yu,et al.  Learning a Discriminative Feature Network for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Jonathan G. Fiscus,et al.  MFC Datasets: Large-Scale Benchmark Datasets for Media Forensic Challenge Evaluation , 2019, 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[10]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[11]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[12]  Stefan Winkler,et al.  COVERAGE — A novel database for copy-move forgery detection , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[13]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[14]  Luisa Verdoliva,et al.  Media Forensics and DeepFakes: An Overview , 2020, IEEE Journal of Selected Topics in Signal Processing.

[15]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[16]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Zhenheng Yang,et al.  SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization , 2020, ECCV.

[18]  Belhassen Bayar,et al.  Constrained Convolutional Neural Networks: A New Approach Towards General Purpose Image Manipulation Detection , 2018, IEEE Transactions on Information Forensics and Security.

[19]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Rafael Grompone von Gioi,et al.  An Adaptive Neural Network for Unsupervised Mosaic Consistency Analysis in Image Forensics , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  FridrichJessica,et al.  Rich Models for Steganalysis of Digital Images , 2012 .

[22]  Xirong Li,et al.  Learn to Segment Retinal Lesions and Beyond , 2019, ArXiv.

[23]  C.-C. Jay Kuo,et al.  Image Splicing Localization using a Multi-task Fully Convolutional Network (MFCN) , 2017, J. Vis. Commun. Image Represent..

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Premkumar Natarajan,et al.  ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  B. S. Manjunath,et al.  Exploiting Spatial Structure for Localizing Manipulated Image Regions , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Jiwu Huang,et al.  Localization of Deep Inpainting Using High-Pass Fully Convolutional Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[30]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[31]  Gaël MAHFOUDI,et al.  DEFACTO: Image and Face Manipulation Dataset , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).