Computer Graphics Identification Combining Convolutional and Recurrent Neural Networks

In this letter, a deep-learning-based pipeline is proposed to distinguish photographics (PGs) from computer-graphics (CGs) combining convolutional neural network (CNN) and recurrent neural network (RNN). In the preprocessing stage, the color space transformation and the Schmid filter bank are utilized to extract chrominance and luminance components, which suppress the irrelevant information of various image contents for the CG identification task. Then, a dual-path CNN architecture is designed to learn joint feature representations of local patches for exploiting their color and texture characteristics. To extract the global artifact, the directed acyclic graph RNN is applied to model the spatial dependence of local patterns. Finally, the output score of RNN is used to identify the input sample. The CG/PG dataset is constructed by collecting samples from the Internet. Experimental results show that the proposed framework can outperform state-of-the-art methods on identification ability of CGs, especially for images with low resolution.

[1]  Siwei Lyu,et al.  Higher-order Wavelet Statistics and their Application to Digital Forensics , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[2]  Yun Q. Shi,et al.  Identifying Computer Graphics using HSV Color Model and Statistical Moments of Characteristic Functions , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[3]  Feng Pan,et al.  Discriminating between photorealistic computer graphics and natural images using fractal geometry , 2009, Science in China Series F: Information Sciences.

[4]  Jianhua Li,et al.  Identifying computer generated and digital camera images using fractional lower order moments , 2009, 2009 4th IEEE Conference on Industrial Electronics and Applications.

[5]  Weidong Min,et al.  DETECTING THE COMPOSITE OF PHOTOGRAPHIC IMAGE AND COMPUTER GENERATED IMAGE COMBINING WITH COLOR , TEXTURE AND SHAPE FEATURE , 2013 .

[6]  Anderson Rocha,et al.  Computer generated images vs. digital photographs: A synergetic feature and classifier combination approach , 2013, J. Vis. Commun. Image Represent..

[7]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[8]  Siwei Lyu,et al.  How realistic is photorealistic? , 2005, IEEE Transactions on Signal Processing.

[9]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Belhassen Bayar,et al.  On the robustness of constrained convolutional neural networks to JPEG post-compression for image resampling detection , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Yun Q. Shi,et al.  Distinguishing Computer Graphics from Photographic Images Using Local Binary Patterns , 2012, IWDW.

[13]  Jianru Xue,et al.  A statistical feature based approach to distinguish PRCG from photographs , 2014, Comput. Vis. Image Underst..

[14]  Paolo Bestagini,et al.  First Steps Toward Camera Model Identification With Convolutional Neural Networks , 2016, IEEE Signal Processing Letters.

[15]  Ismail Avcibas,et al.  Differentiating Between Images Using Wavelet-Based Transforms: A Comparative Study , 2011, IEEE Transactions on Information Forensics and Security.

[16]  Gang Wang,et al.  DAG-Recurrent Neural Networks for Scene Labeling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[18]  H. P. Chen,et al.  Detecting computer generated images based on local ternary count , 2016, Pattern Recognition and Image Analysis.

[19]  Shiguo Lian,et al.  Forensics feature analysis in quaternion wavelet domain for distinguishing photographic images and computer graphics , 2017, Multimedia Tools and Applications.

[20]  Fei Peng,et al.  Identification of Natural Images and Computer‐Generated Graphics Based on Statistical and Textural Features , 2015, Journal of forensic sciences.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[23]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[24]  Fei Peng,et al.  Discrimination of natural images and computer generated graphics based on multi-fractal and regression analysis , 2017 .

[25]  Dong-Ming Yan,et al.  Distinguishing Between Natural and Computer-Generated Images Using Convolutional Neural Networks , 2018, IEEE Transactions on Information Forensics and Security.

[26]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Jong-Uk Hou,et al.  Identifying photorealistic computer graphics using convolutional neural networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[28]  Min Wu,et al.  Information Forensics: An Overview of the First Decade , 2013, IEEE Access.

[29]  James Martens,et al.  Deep learning via Hessian-free optimization , 2010, ICML.

[30]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[31]  Stacy Holman Jones Torch , 1999 .

[32]  Junichi Yamagishi,et al.  Distinguishing computer graphics from natural images using convolution neural networks , 2017, 2017 IEEE Workshop on Information Forensics and Security (WIFS).

[33]  Paolo Bestagini,et al.  Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks , 2017, J. Vis. Commun. Image Represent..

[34]  Shih-Fu Chang,et al.  Physics-motivated features for distinguishing photographic images and computer graphics , 2005, ACM Multimedia.

[35]  Z. Jane Wang,et al.  Median Filtering Forensics Based on Convolutional Neural Networks , 2015, IEEE Signal Processing Letters.

[36]  Jiangqun Ni,et al.  A deep learning approach to detection of splicing and copy-move forgeries in images , 2016, 2016 IEEE International Workshop on Information Forensics and Security (WIFS).

[37]  Alex ChiChung Kot,et al.  Image Recapture Detection with Convolutional and Recurrent Neural Networks , 2017, Media Watermarking, Security, and Forensics.

[38]  Cordelia Schmid,et al.  Constructing models for content-based image retrieval , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.