Pigment Unmixing of Hyperspectral Images of Paintings Using Deep Neural Networks

In this paper, the problem of automatic nonlinear unmixing of hyperspectral reflectance data using works of art as test cases is described. We use a deep neural network to decompose a given spectrum quantitatively to the abundance values of pure pigments. We show that adding another step to identify the constituent pigments of a given spectrum leads to more accurate unmixing results. Towards this, we use another deep neural network to identify pigments first and integrate this information to different layers of the network used for pigment unmixing. As a test set, the hyperspectral images of a set of mock-up paintings consisting of a broad palette of pigment mixtures, and pure pigment exemplars, were measured. The results of the algorithm on the mock-up test set are reported and analyzed.

[1]  Shihong Du,et al.  Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Aggelos K. Katsaggelos,et al.  Automatic pigment identification on roman Egyptian paintings by using sparse modeling of hyperspectral images , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[3]  Shanjun Mao,et al.  Spectral–spatial classification of hyperspectral images using deep convolutional neural networks , 2015 .

[4]  Nikolaos Doulamis,et al.  Deep supervised learning for hyperspectral data classification through convolutional neural networks , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[5]  Emeline Pouyet,et al.  Nonlinear Unmixing of Hyperspectral Datasets for the Study of Painted Works of Art. , 2018, Angewandte Chemie.

[6]  Jan A Snyman,et al.  Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms , 2005 .

[7]  Paola Ricciardi,et al.  Use of imaging spectroscopy, fiber optic reflectance spectroscopy, and X-ray fluorescence to map and identify pigments in illuminated manuscripts , 2014 .

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Rui Guo,et al.  Hyperspectral image unmixing using autoencoder cascade , 2015, 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[11]  Roy S. Berns,et al.  Single-Constant Simplification of Kubelka-Munk Turbid-Media Theory for Paint Systems—A Review , 2007 .

[12]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[13]  R. Berns,et al.  Evaluating Single- and Two-Constant Kubelka-Munk Turbid Media Theory for Instrumental-Based Inpainting , 2007 .

[14]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[15]  Przemysław Głomb,et al.  Automatic pigment identification from hyperspectral data , 2018 .

[16]  Hervé Abdi.  The Method of Least Squares , 2007 .

[17]  P. Kubelka,et al.  New Contributions to the Optics of Intensely Light-Scattering Materials. Part I , 1948 .

[18]  M. Menu,et al.  Quantitative characterisation of pigment mixtures used in art by fibre-optics diffuse-reflectance spectroscopy , 2006 .