Tampered and Computer-Generated Face Images Identification Based on Deep Learning

Image forgery is an active topic in digital image tampering that is performed by moving a region from one image into another image, combining two images to form one image, or retouching an image. Moreover, recent developments of generative adversarial networks (GANs) that are used to generate human facial images have made it more challenging for even humans to detect the tampered one. The spread of those images on the internet can cause severe ethical, moral, and legal issues if the manipulated images are misused. As a result, much research has been conducted to detect facial image manipulation based on applying machine learning algorithms on tampered face datasets in the last few years. This paper introduces a deep learning-based framework that can identify manipulated facial images and GAN-generated images. It is comprised of multiple convolutional layers, which can efficiently extract features using multi-level abstraction from tampered regions. In addition, a data-based approach, cost-sensitive learning-based approach (class weight), and ensemble-based approach (eXtreme Gradient Boosting) is applied to the proposed model to deal with the imbalanced data problem (IDP). The superiority of the proposed model that deals with an IDP is verified using a tampered face dataset and a GAN-generated face dataset under various scenarios. Experimental results proved that the proposed framework outperformed existing expert systems, which has been used for identifying manipulated facial images and GAN-generated images in terms of computational complexity, area under the curve (AUC), and robustness. As a result, the proposed framework inspires the development of research on image forgery identification and enables the potential to integrate these models into practical applications, which require tampered facial image detection.

[1]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[2]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[3]  Tom A. B. Snijders,et al.  Social Network Analysis , 2011, International Encyclopedia of Statistical Science.

[4]  Qiang Ji,et al.  Robust Facial Landmark Detection Under Significant Head Poses and Occlusion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[6]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Francisco Herrera,et al.  Cost-sensitive linguistic fuzzy rule based classification systems under the MapReduce framework for imbalanced big data , 2015, Fuzzy Sets Syst..

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Chang Ouk Kim,et al.  An Incremental Clustering-Based Fault Detection Algorithm for Class-Imbalanced Process Data , 2015, IEEE Transactions on Semiconductor Manufacturing.

[10]  Sanjeev Jain,et al.  Survey on Keypoint Based Copy-move Forgery Detection Methods on Image , 2016 .

[11]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[12]  Sattar Hashemi,et al.  To Combat Multi-Class Imbalanced Problems by Means of Over-Sampling Techniques , 2016, IEEE Transactions on Knowledge and Data Engineering.

[13]  Anderson Rocha,et al.  Illuminant-Based Transformed Spaces for Image Forensics , 2016, IEEE Transactions on Information Forensics and Security.

[14]  Francisco Herrera,et al.  Fuzzy rough classifiers for class imbalanced multi-instance data , 2016, Pattern Recognit..

[15]  Ali Ahmadi,et al.  A novel forensic image analysis tool for discovering double JPEG compression clues , 2017, Multimedia Tools and Applications.

[16]  Jun-Hai Zhai,et al.  The classification of imbalanced large data sets based on MapReduce and ensemble of ELM classifiers , 2015, International Journal of Machine Learning and Cybernetics.

[17]  Chih-Fong Tsai,et al.  Clustering-based undersampling in class-imbalanced data , 2017, Inf. Sci..

[18]  Larry S. Davis,et al.  Two-Stream Neural Networks for Tampered Face Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Leandro dos Santos Coelho,et al.  Image forgery detection by semi-automatic wavelet soft-Thresholding with error level analysis , 2017, Expert Syst. Appl..

[20]  Elad Hoffer,et al.  Train longer, generalize better: closing the generalization gap in large batch training of neural networks , 2017, NIPS.

[21]  Yijing Li,et al.  Learning from class-imbalanced data: Review of methods and applications , 2017, Expert Syst. Appl..

[22]  Jean-Luc Dugelay,et al.  Face aging with conditional generative adversarial networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[23]  Yao Zhao,et al.  Copy-Move Forgery Localization Using Convolutional Neural Networks and CFA Features , 2018, Int. J. Digit. Crime Forensics.

[24]  Hyeonjoon Moon,et al.  Deep Learning Based Computer Generated Face Identification Using Convolutional Neural Network , 2018, Applied Sciences.

[25]  Tal Hassner,et al.  Facial Landmark Detection with Tweaked Convolutional Neural Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[27]  Mohammed Bennamoun,et al.  Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Hung Nguyen-Xuan,et al.  A novel analysis-prediction approach for geometrically nonlinear problems using group method of data handling , 2019, Computer Methods in Applied Mechanics and Engineering.

[29]  Hyeonjoon Moon,et al.  Face image manipulation detection based on a convolutional neural network , 2019, Expert Syst. Appl..

[30]  Chien H. Thai,et al.  NURBS-based postbuckling analysis of functionally graded carbon nanotube-reinforced composite shells , 2019, Computer Methods in Applied Mechanics and Engineering.

[31]  Tan N. Nguyen,et al.  A novel data-driven nonlinear solver for solid mechanics using time series forecasting , 2020 .

[32]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.