Detection of Subtle Bruises on Winter Jujube Using Hyperspectral Imaging With Pixel-Wise Deep Learning Method

Winter jujubes get bruised easily during harvest and transportation. In order to detect subtle bruises on winter jujubes in a more efficient way, a rapid and accurate technique, hyperspectral imaging was used. Near-infrared reflectance (NIR) and visible/near-infrared reflectance (Vis-NIR) hyperspectral imaging at the spectral region of 874-1734 nm and 380-1030 nm, respectively, were applied in this study. The hyperspectral images of winter jujubes from four geographical origins were acquired. Pixel-wise spectra were extracted and preprocessed; pixel-wise principal component analysis (PCA) was used to conduct a qualitative analysis. Accuracy, true positive rate (TPR) and false positive rate (FPR) were utilized to compare the efficiency of the models. Support vector machine (SVM), logistic regression (LR) and a deep learning method, and convolutional neural network (CNN) were used to build pixel-wise classification models based on single or all geographical origins for quantitative analyses. All the models using NIR spectra obtained decent results with accuracies in the range of 90–100%, and TPRs and FPRs close to 1 and 0, respectively. Compared with the other two methods using Vis-NIR spectra, the CNN model based on all geographical origins got the best performance with most of the accuracies surpassing 85%. For Vis-NIR spectra and NIR spectra, the overall time efficiency for modeling and prediction of CNN was at an intermediate level among the three models. The short prediction time of CNN indicated that CNN had the potential for real-time detection. The prediction maps obtained by the CNN models indicated that the color information and geographical origins could affect the detection performance. The overall results demonstrated the promising potential for detecting subtle bruises on winter jujubes using pixel-wise spectra extracted from the hyperspectral images at the two spectral ranges with the deep learning method. The results in this study would help to develop an online winter jujube bruises detection system in the future.

[1]  Chu Zhang,et al.  Application of Visible and Near-Infrared Hyperspectral Imaging to Determine Soluble Protein Content in Oilseed Rape Leaves , 2015, Sensors.

[2]  Fei Liu,et al.  Mid-infrared spectroscopy combined with chemometrics to detect Sclerotinia stem rot on oilseed rape (Brassica napus L.) leaves , 2017, Plant Methods.

[3]  M. I. Zhang,et al.  Apple bruise assessment through electrical impedance measurements , 1993 .

[4]  P. Baranowski,et al.  Detection of early bruises in apples using hyperspectral data and thermal imaging , 2012 .

[5]  Chu Zhang,et al.  Application of Near-Infrared Hyperspectral Imaging to Detect Sulfur Dioxide Residual in the Fritillaria thunbergii Bulbus Treated by Sulfur Fumigation , 2017 .

[6]  Q. Yang,et al.  An approach to apple surface feature detection by machine vision , 1994 .

[7]  Shanjun Mao,et al.  Spectral–spatial classification of hyperspectral images using deep convolutional neural networks , 2015 .

[8]  Zhaohui Xue,et al.  Effect of abscisic acid (ABA) and chitosan/nano-silica/sodium alginate composite film on the color development and quality of postharvest Chinese winter jujube (Zizyphus jujuba Mill. cv. Dongzao). , 2019, Food chemistry.

[9]  Sarun Sumriddetchkajorn,et al.  Hyperspectral imaging-based credit card verifier structure with adaptive learning. , 2008, Applied optics.

[10]  Guolan Lu,et al.  Medical hyperspectral imaging: a review , 2014, Journal of biomedical optics.

[11]  J. Qin,et al.  Detection of citrus canker using hyperspectral reflectance imaging with spectral information divergence , 2009 .

[12]  Wenyi Tan,et al.  Detecting and classifying minor bruised potato based on hyperspectral imaging , 2018, Chemometrics and Intelligent Laboratory Systems.

[13]  Margarita Osadchy,et al.  Deep Convolutional Neural Networks for Raman Spectrum Recognition: A Unified Solution , 2017, The Analyst.

[14]  Paul M. Mather,et al.  Assessment of the effectiveness of support vector machines for hyperspectral data , 2004, Future Gener. Comput. Syst..

[15]  Jun Wang,et al.  Predictions of acidity, soluble solids and firmness of pear using electronic nose technique , 2008 .

[16]  Qingshan Liu,et al.  Bidirectional-Convolutional LSTM Based Spectral-Spatial Feature Learning for Hyperspectral Image Classification , 2017, Remote. Sens..

[17]  Elena Marchiori,et al.  Convolutional neural networks for vibrational spectroscopic data analysis. , 2017, Analytica chimica acta.

[18]  Santosh Lohumi,et al.  Near-infrared hyperspectral imaging system coupled with multivariate methods to predict viability and vigor in muskmelon seeds , 2016 .

[19]  Nuri N. Mohsenin.  Structure, physical characteristics, and rheological properties , 1968 .

[20]  Wei Liu,et al.  Nondestructive determination of transgenic Bacillus thuringiensis rice seeds (Oryza sativa L.) using multispectral imaging and chemometric methods. , 2014, Food chemistry.

[21]  Marena Manley,et al.  Classification of Maize Kernel Hardness Using near Infrared Hyperspectral Imaging , 2012 .

[22]  Hanping Mao,et al.  A Method for Rapid Identification of Rice Origin by Hyperspectral Imaging Technology , 2017 .

[23]  Wenqian Huang,et al.  Data Fusion of Two Hyperspectral Imaging Systems with Complementary Spectral Sensing Ranges for Blueberry Bruising Detection , 2018, Sensors.

[24]  Lorenzo Bruzzone,et al.  Deep feature representation for hyperspectral image classification , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[25]  Yibin Ying,et al.  DeepSpectra: An end-to-end deep learning approach for quantitative spectral analysis. , 2019, Analytica chimica acta.

[26]  Ning Wang,et al.  Early detection of apple bruises on different background colors using hyperspectral imaging , 2008 .

[27]  Rick Archibald,et al.  Feature Selection and Classification of Hyperspectral Images With Support Vector Machines , 2007, IEEE Geoscience and Remote Sensing Letters.

[28]  Hongbin Pu,et al.  Prediction of Color and pH of Salted Porcine Meats Using Visible and Near-Infrared Hyperspectral Imaging , 2014, Food and Bioprocess Technology.

[29]  Laijun Sun,et al.  Pixel based bruise region extraction of apple using Vis-NIR hyperspectral imaging , 2018, Comput. Electron. Agric..

[30]  Zhenjie Xiong,et al.  Application of Hyperspectral Imaging for Prediction of Textural Properties of Maize Seeds with Different Storage Periods , 2015, Food Analytical Methods.

[31]  Daniel E. Guyer,et al.  Near-infrared hyperspectral reflectance imaging for detection of bruises on pickling cucumbers , 2006, Computers and Electronics in Agriculture.

[32]  Linmi Tao,et al.  Efficient Deep Auto-Encoder Learning for the Classification of Hyperspectral Images , 2016, 2016 International Conference on Virtual Reality and Visualization (ICVRV).

[33]  Chu Zhang,et al.  Variety Identification of Single Rice Seed Using Hyperspectral Imaging Combined with Convolutional Neural Network , 2018 .

[34]  Akira Uemura,et al.  Determination of Seed Soundness in Conifers Cryptomeria japonica and Chamaecyparis obtusa Using Narrow-Multiband Spectral Imaging in the Short-Wavelength Infrared Range , 2015, PloS one.

[35]  Berrin A. Yanikoglu,et al.  Deep Learning With Attribute Profiles for Hyperspectral Image Classification , 2016, IEEE Geoscience and Remote Sensing Letters.

[36]  Chu Zhang,et al.  Rapid and non-destructive measurement of spinach pigments content during storage using hyperspectral imaging with chemometrics , 2017 .

[37]  Lalit Mohan Kandpal,et al.  High speed measurement of corn seed viability using hyperspectral imaging , 2016 .

[38]  Chu Zhang,et al.  Discrimination of Chrysanthemum Varieties Using Hyperspectral Imaging Combined with a Deep Convolutional Neural Network , 2018, Molecules.

[39]  Chu Zhang,et al.  Identification of coffee bean varieties using hyperspectral imaging: influence of preprocessing methods and pixel-wise spectra analysis , 2018, Scientific Reports.

[40]  Jiewen Zhao,et al.  Detection of Bruise on Pear by Hyperspectral Imaging Sensor with Different Classification Algorithms , 2010 .

[41]  Bor-Chen Kuo,et al.  A Kernel-Based Feature Selection Method for SVM With RBF Kernel for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[42]  Chu Zhang,et al.  Variety Identification of Raisins Using Near-Infrared Hyperspectral Imaging , 2018, Molecules.

[43]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.

[44]  Huanda Lu,et al.  Deep-learning-based regression model and hyperspectral imaging for rapid detection of nitrogen concentration in oilseed rape (Brassica napus L.) leaf , 2018 .

[45]  Yong He,et al.  Application of hyperspectral imaging and chemometrics for variety classification of maize seeds , 2018, RSC advances.

[46]  Shanjun Mao,et al.  A deep learning framework for hyperspectral image classification using spatial pyramid pooling , 2016 .

[47]  Yang Tao,et al.  Building a rule-based machine-vision system for defect inspection on apple sorting and packing lines , 1999 .

[48]  Shuxiang Fan,et al.  Detection of blueberry internal bruising over time using NIR hyperspectral reflectance imaging with optimum wavelengths , 2017 .

[49]  Chu Zhang,et al.  Identification of Maize Kernel Vigor under Different Accelerated Aging Times Using Hyperspectral Imaging , 2018, Molecules.

[50]  Xinjie Yu,et al.  Nondestructive Freshness Discriminating of Shrimp Using Visible/Near-Infrared Hyperspectral Imaging Technique and Deep Learning Algorithm , 2018, Food Analytical Methods.

[51]  David Bergström,et al.  Hyperspectral image analysis using deep learning — A review , 2016, 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA).