Deep learning for automated forgery detection in hyperspectral document images

Abstract. Deep learning is revolutionizing the already rapidly developing field of computer vision. The convolutional neural network (CNN) is a state-of-the-art deep learning tool that learns high level features directly from a huge dataset of labeled images. In document image processing, ink analysis allows for determination of ink age and forgery and identification of pen or writer. The spectral information of inks in hyperspectral document images provides valuable information about the underlying material and thus helps in identification and discrimination of inks based on their unique spectral signatures even if they have the same color. Ink mismatch detection is a key step in document forgery detection. Although various ink mismatch detection techniques are available in the recent literature, there is a constant need for development of more accurate and effective methods to empower automated document forgery detection. A state-of-the-art deep learning method for ink mismatch detection in hyperspectral document images is proposed. The spectral responses of ink pixels are extracted from a hyperspectral document image, reshaped to a CNN-friendly image format and fed to the CNN for classification. The proposed method effectively identifies different ink types in a hyperspectral document image for forgery detection and achieves an overall accuracy of 98.2% for blue and 88% for black inks, which is the highest accuracy among the latest techniques of ink mismatch detection on the UWA Writing Ink Hyperspectral Images (WIHSI) database and differentiates between the highest number of inks mixed in unbalanced proportions in a hyperspectral document image. Furthermore, a detailed discussion on selection of appropriate CNN architecture and classification results are presented in this paper along with comparison with the former methods of ink mismatch detection. This research opens a new window for research on automated forgery detection in hyperspectral document images using deep learning.

[1]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Tara N. Sainath,et al.  Deep Convolutional Neural Networks for Large-scale Speech Tasks , 2015, Neural Networks.

[3]  Kristalia Melessanaki,et al.  Laser induced breakdown spectroscopy and hyper-spectral imaging analysis of pigments on an illuminated manuscript , 2001 .

[4]  Eric B Brauns,et al.  Fourier Transform Hyperspectral Visible Imaging and the Nondestructive Analysis of Potentially Fraudulent Documents , 2006, Applied spectroscopy.

[5]  Ajmal S. Mian,et al.  Localized forgery detection in hyperspectral document images , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[6]  Nicolas Papadakis,et al.  A novel hyper-spectral imaging apparatus for the non-destructive analysis of objects of artistic and historic value , 2003 .

[7]  Thomas M. Breuel,et al.  Efficient implementation of local adaptive thresholding techniques using integral images , 2008, Electronic Imaging.

[8]  Miguel Angel Ferrer-Ballester,et al.  The use of hyperspectral analysis for ink identification in handwritten documents , 2014, 2014 International Carnahan Conference on Security Technology (ICCST).

[9]  Yansheng Li,et al.  Unsupervised Spectral–Spatial Feature Learning With Stacked Sparse Autoencoder for Hyperspectral Imagery Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[10]  Katrin Franke,et al.  Ink-deposition model: the relation of writing and ink deposition processes , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[11]  Ajmal S. Mian,et al.  Hyperspectral Imaging for Ink Mismatch Detection , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[12]  David A. Landgrebe,et al.  Information Extraction Principles and Methods for Multispectral and Hyperspectral Image Data , 1999 .

[13]  Hamid Saeed Khan,et al.  Modern Trends in Hyperspectral Image Analysis: A Review , 2018, IEEE Access.

[14]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[15]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[16]  Mohamed Cheriet,et al.  Constrained Energy Maximization and Self-Referencing Method for Invisible Ink Detection from Multispectral Historical Document Images , 2014, 2014 22nd International Conference on Pattern Recognition.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[19]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Bor-Chen Kuo,et al.  Feature Mining for Hyperspectral Image Classification , 2013, Proceedings of the IEEE.

[21]  Khurram Khurshid,et al.  Breast cancer detection in mammograms using convolutional neural network , 2018, 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET).

[22]  Melanie Gau,et al.  Enhancement of Multispectral Images of Degraded Documents by Employing Spatial Information , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[23]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[24]  Sinan Kalkan,et al.  Deep Hierarchies in the Primate Visual Cortex: What Can We Learn for Computer Vision? , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Maria Fernanda Pimentel,et al.  Near infrared hyperspectral imaging for forensic analysis of document forgery. , 2014, The Analyst.

[26]  Marvin E. Klein,et al.  Quantitative Hyperspectral Reflectance Imaging , 2008, Sensors.

[27]  Michael S. Brown,et al.  Visual enhancement of old documents with hyperspectral imaging , 2011, Pattern Recognit..

[28]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Khurram Khurshid,et al.  Automated Forgery Detection in Multispectral Document Images Using Fuzzy Clustering , 2018, 2018 13th IAPR International Workshop on Document Analysis Systems (DAS).

[30]  Khurram Khurshid,et al.  Towards Automated Ink Mismatch Detection in Hyperspectral Document Images , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[31]  Qi Li,et al.  Hyperspectral Imagery Classification Using Sparse Representations of Convolutional Neural Network Features , 2016, Remote. Sens..

[32]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[33]  M. Imran,et al.  Benchmark dataset for offline handwritten character recognition , 2017, 2017 13th International Conference on Emerging Technologies (ICET).

[34]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[35]  Katrin Franke,et al.  Ink texture analysis for writer identification , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.