Artificial Intelligence for Pigment Classification Task in the Short-Wave Infrared Range

Hyperspectral reflectance imaging in the short-wave infrared range (SWIR, “extended NIR”, ca. 1000 to 2500 nm) has proven to provide enhanced characterization of paint materials. However, the interpretation of the results remains challenging due to the intrinsic complexity of the SWIR spectra, presenting both broad and narrow absorption features with possible overlaps. To cope with the high dimensionality and spectral complexity of such datasets acquired in the SWIR domain, one data treatment approach is tested, inspired by innovative development in the cultural heritage field: the use of a pigment spectral database (extracted from model and historical samples) combined with a deep neural network (DNN). This approach allows for multi-label pigment classification within each pixel of the data cube. Conventional Spectral Angle Mapping and DNN results obtained on both pigment reference samples and a Buddhist painting (thangka) are discussed.

[1]  K. Shadan,et al.  Available online: , 2012 .

[2]  Paola Ricciardi,et al.  Visible and infrared imaging spectroscopy of paintings and improved reflectography , 2016, Heritage Science.

[3]  Aggelos K. Katsaggelos,et al.  Pigment Unmixing of Hyperspectral Images of Paintings Using Deep Neural Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  C. Sabbarese,et al.  Combination of noninvasive imaging techniques to characterize pigments in Buddhist thangka paintings , 2020, X-Ray Spectrometry.

[5]  Matthias Alfeld,et al.  Recent developments in spectroscopic imaging techniques for historical paintings - A review , 2017 .

[6]  Marcello Picollo,et al.  Hyper-Spectral Imaging Technique in the Cultural Heritage Field: New Possible Scenarios , 2020, Sensors.

[7]  J. Nobbs.  Kubelka—Munk Theory and the Prediction of Reflectance , 2008 .

[8]  P. Ricciardi,et al.  21 The Five Colours of Art: Non-invasive Analysis of Pigments in Tibetan Prints and Manuscripts , 2016 .

[9]  M. Mestre,et al.  Mapping pigments and binders in 15th century Gothic works of art using a combination of visible and near infrared hyperspectral imaging , 2020, Microchemical Journal.

[10]  Przemysław Głomb,et al.  Automatic pigment identification from hyperspectral data , 2018 .

[11]  D. Jackson,et al.  Tibetan Thangka Painting: Methods and Materials , 1984 .

[12]  P. Ricciardi,et al.  ‘It's not easy being green’: a spectroscopic study of green pigments used in illuminated manuscripts , 2013 .

[15]  Lifu Zhang,et al.  Shortwave Infrared Imaging Spectroscopy for Analysis of Ancient Paintings , 2017, Applied spectroscopy.

[16]  K. A. Dooley,et al.  Near-UV to mid-IR reflectance imaging spectroscopy of paintings on the macroscale , 2019, Science Advances.

[17]  D. Messinger,et al.  An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence , 2020, Heritage Science.

[18]  Silvia Rita Amato,et al.  A Preliminary Study on the Differentiation of Linseed and Poppy Oil Using Principal Component Analysis Methods Applied to Fiber Optics Reflectance Spectroscopy and Diffuse Reflectance Imaging Spectroscopy , 2020, Sensors.

[19]  Nonlinear Mixing Characteristics of Reflectance Spectra of Typical Mineral Pigments , 2021 .

[20]  L. Bokobza.  Near Infrared Spectroscopy , 1998 .

[21]  Paola Ricciardi,et al.  Mapping of egg yolk and animal skin glue paint binders in Early Renaissance paintings using near infrared reflectance imaging spectroscopy. , 2013, The Analyst.

[22]  C. Miliani,et al.  FT-NIR spectroscopy for non-invasive identification of natural polymers and resins in easel paintings , 2009, Analytical and bioanalytical chemistry.

[23]  Y. Ozaki,et al.  Advances in Molecular Structure and Interaction Studies Using Near-Infrared Spectroscopy. , 2015, Chemical reviews.

[24]  P. R. Meneses,et al.  Spectral Correlation Mapper ( SCM ) : An Improvement on the Spectral Angle Mapper ( SAM ) , 2000 .

[25]  Marcello Picollo,et al.  Reflectance Hyperspectral Imaging for Investigation of Works of Art: Old Master Paintings and Illuminated Manuscripts. , 2016, Accounts of chemical research.

[26]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[27]  Emeline Pouyet,et al.  Nonlinear Unmixing of Hyperspectral Datasets for the Study of Painted Works of Art. , 2018, Angewandte Chemie.

[28]  V. A. Solé,et al.  A multiplatform code for the analysis of energy-dispersive X-ray fluorescence spectra , 2007 .

[29]  Norimichi Tsumura,et al.  Estimating Pigment Concentrations from Spectral Images Using an Encoder‐Decoder Neural Network , 2020 .