Noise reduction in the spectral domain of hyperspectral images using denoising autoencoder methods

Abstract Denoising of spectra has been a great challenge in hyperspectral image analysis. Near-infrared hyperspectral images of milk powder, rice flour and soybean flour were acquired and denoising in the spectral domain were studied. Noise free spectra and noises were simulated based on sample pixel-wise spectra. The noisy spectra with signal to noise ratio (SNR) around 45 ​dB (similar to real pixel-wise spectra) were simulated. The simulated noisy spectra were preprocessed by traditional methods as moving average smoothing (MAS), Savitzky-Golay smoothing (SGS), wavelet transform (WT) and empirical mode decomposition (EMD). The basic denoising autoencoder (DAE-1) and the stacked DAE (DAE-2) were studied for denoising. The noisy spectra with SNR around 35 ​dB and 55 ​dB were further simulated to explore the effectiveness of DAE based methods. DAE-1 and DAE-2 performed better than the other methods, with higher SNR, lower mean squared error (MSE) and mean absolute error (MAE). The developed DAE methods were applied to real-world pixel-wise spectra with good performances. The overall results proved the feasibility of using DAE based methods for noise reduction in the spectral domain of hyperspectral images, and the DAE based methods have great potential to be extended to spectral denoising of other vibrational spectroscopy techniques.

[1]  Chunsheng Cai,et al.  Different Discrete Wavelet Transforms Applied to Denoising Analytical Data , 1998, J. Chem. Inf. Comput. Sci..

[2]  Elena Marchiori,et al.  Convolutional neural networks for vibrational spectroscopic data analysis. , 2017, Analytica chimica acta.

[3]  Paul J. Williams,et al.  Classification of maize kernels using NIR hyperspectral imaging. , 2016, Food chemistry.

[4]  Jan F. Humplík,et al.  Automated phenotyping of plant shoots using imaging methods for analysis of plant stress responses – a review , 2015, Plant Methods.

[5]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[6]  Salim Lahmiri,et al.  Image denoising in bidimensional empirical mode decomposition domain: the role of Student's probability distribution function. , 2016, Healthcare technology letters.

[7]  Qiang Zhang,et al.  Hyperspectral Image Denoising Employing a Spatial–Spectral Deep Residual Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Chu Zhang,et al.  Identification of coffee bean varieties using hyperspectral imaging: influence of preprocessing methods and pixel-wise spectra analysis , 2018, Scientific Reports.

[9]  Margarita Ruiz-Altisent,et al.  Examination of the quality of spinach leaves using hyperspectral imaging , 2013 .

[10]  Andrew P French,et al.  Hyperspectral image analysis techniques for the detection and classification of the early onset of plant disease and stress , 2017, Plant Methods.

[11]  Margarita Ruiz-Altisent,et al.  Monitoring spinach shelf-life with hyperspectral image through packaging films , 2013 .

[12]  B. Cho,et al.  In-Process Control Assay of Pharmaceutical Microtablets Using Hyperspectral Imaging Coupled with Multivariate Analysis. , 2016, Analytical Chemistry.

[13]  Yong He,et al.  Discriminating varieties of tea plant based on Vis/NIR spectral characteristics and using artificial neural networks , 2008 .

[14]  Chu Zhang,et al.  Variety Identification of Single Rice Seed Using Hyperspectral Imaging Combined with Convolutional Neural Network , 2018 .

[15]  Chu Zhang,et al.  Identification of Soybean Varieties Using Hyperspectral Imaging Coupled with Convolutional Neural Network , 2019, Sensors.

[16]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[17]  Takehisa Yairi,et al.  A review on the application of deep learning in system health management , 2018, Mechanical Systems and Signal Processing.

[18]  Lalit Mohan Kandpal,et al.  Hyperspectral Reflectance Imaging Technique for Visualization of Moisture Distribution in Cooked Chicken Breast , 2013, Sensors.

[19]  C. De Bleye,et al.  Data processing of vibrational chemical imaging for pharmaceutical applications. , 2014, Journal of pharmaceutical and biomedical analysis.

[20]  Yidan Bao,et al.  Hyperspectral imaging for seed quality and safety inspection: a review , 2019, Plant Methods.

[21]  R. Bonner,et al.  Application of wavelet transforms to experimental spectra : Smoothing, denoising, and data set compression , 1997 .

[22]  Da-Wen Sun,et al.  Advanced Techniques for Hyperspectral Imaging in the Food Industry: Principles and Recent Applications. , 2019, Annual review of food science and technology.

[23]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[24]  Jun-Hu Cheng,et al.  NIR hyperspectral imaging with multivariate analysis for measurement of oil and protein contents in peanut varieties , 2017, Analytical Methods.

[25]  Yunsong Li,et al.  Hyperspectral Imagery Denoising by Deep Learning With Trainable Nonlinearity Function , 2017, IEEE Geoscience and Remote Sensing Letters.

[26]  Marcus Nagle,et al.  Prediction mapping of physicochemical properties in mango by hyperspectral imaging , 2017 .

[27]  Fei Liu,et al.  Application of Deep Learning in Food: A Review. , 2019, Comprehensive reviews in food science and food safety.

[28]  Dan Savastru,et al.  Hyperspectral Imaging in the Medical Field: Present and Future , 2014 .

[29]  Dragica Radosav,et al.  Deep Learning and Medical Diagnosis: A Review of Literature , 2018, Multimodal Technol. Interact..

[30]  Lalit Mohan Kandpal,et al.  High speed measurement of corn seed viability using hyperspectral imaging , 2016 .

[31]  Arun Sharma,et al.  A Review on the Application of Deep Learning in Legal Domain , 2019, AIAI.

[32]  Weiwei Cheng,et al.  Development of simplified models for nondestructive hyperspectral imaging monitoring of TVB-N contents in cured meat during drying process , 2017 .

[33]  Alejandro C. Olivieri,et al.  Chemometrics coupled to vibrational spectroscopy and spectroscopic imaging for the analysis of solid-phase pharmaceutical products: A brief review on non-destructive analytical methods , 2018, TrAC Trends in Analytical Chemistry.

[34]  William H. Press,et al.  Numerical recipes , 1990 .

[35]  Vincent Baeten,et al.  Hyperspectral Imaging Applications in Agriculture and Agro-Food Product Quality and Safety Control: A Review , 2013 .

[36]  Guolan Lu,et al.  Medical hyperspectral imaging: a review , 2014, Journal of biomedical optics.

[37]  Chu Zhang,et al.  Detection of Subtle Bruises on Winter Jujube Using Hyperspectral Imaging With Pixel-Wise Deep Learning Method , 2019, IEEE Access.