Application of Deep Learning Architectures for Accurate and Rapid Detection of Internal Mechanical Damage of Blueberry Using Hyperspectral Transmittance Data

Deep learning has become a widely used powerful tool in many research fields, although not much so yet in agriculture technologies. In this work, two deep convolutional neural networks (CNN), viz. Residual Network (ResNet) and its improved version named ResNeXt, are used to detect internal mechanical damage of blueberries using hyperspectral transmittance data. The original structure and size of hypercubes are adapted for the deep CNN training. To ensure that the models are applicable to hypercube, we adjust the number of filters in the convolutional layers. Moreover, a total of 5 traditional machine learning algorithms, viz. Sequential Minimal Optimization (SMO), Linear Regression (LR), Random Forest (RF), Bagging and Multilayer Perceptron (MLP), are performed as the comparison experiments. In terms of model assessment, k-fold cross validation is used to indicate that the model performance does not vary with the different combination of dataset. In real-world application, selling damaged berries will lead to greater interest loss than discarding the sound ones. Thus, precision, recall, and F1-score are also used as the evaluation indicators alongside accuracy to quantify the false positive rate. The first three indicators are seldom used by investigators in the agricultural engineering domain. Furthermore, ROC curves and Precision-Recall curves are plotted to visualize the performance of classifiers. The fine-tuned ResNet/ResNeXt achieve average accuracy and F1-score of 0.8844/0.8784 and 0.8952/0.8905, respectively. Classifiers SMO/ LR/RF/Bagging/MLP obtain average accuracy and F1-score of 0.8082/0.7606/0.7314/0.7113/0.7827 and 0.8268/0.7796/0.7529/0.7339/0.7971, respectively. Two deep learning models achieve better classification performance than the traditional machine learning methods. Classification for each testing sample only takes 5.2 ms and 6.5 ms respectively for ResNet and ResNeXt, indicating that the deep learning framework has great potential for online fruit sorting. The results of this study demonstrate the potential of deep CNN application on analyzing the internal mechanical damage of fruit.

[1]  Wengang Zheng,et al.  Potential using of infrared thermal imaging to detect volatile compounds released from decayed grapes , 2017, PloS one.

[2]  Anthony King,et al.  Technology: The Future of Agriculture , 2017, Nature.

[3]  James Patrick Underwood,et al.  Deep fruit detection in orchards , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Sang-Yong Rhee,et al.  Plant Leaf Recognition Using a Convolution Neural Network , 2017, Int. J. Fuzzy Log. Intell. Syst..

[5]  Daniel Sbarbaro,et al.  Hyperspectral transmittance imaging of the shell-free cooked clam Mulinia edulis for parasite detection , 2013 .

[6]  Min Zhang,et al.  Detection of insect-damaged vegetable soybeans using hyperspectral transmittance image , 2013 .

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Shuxiang Fan,et al.  Detection of blueberry internal bruising over time using NIR hyperspectral reflectance imaging with optimum wavelengths , 2017 .

[9]  P. Pathare,et al.  Bruise damage measurement and analysis of fresh horticultural produce-A review , 2014 .

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Da-Wen Sun,et al.  Hyperspectral imaging technique for evaluating food quality and safety during various processes: A review of recent applications , 2017 .

[12]  K. Tu,et al.  Detection of cold injury in peaches by hyperspectral reflectance imaging and artificial neural network. , 2016, Food chemistry.

[13]  Dong Sun Park,et al.  Spectral Analysis of CNN for Tomato Disease Identification , 2017, ICAISC.

[14]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[15]  Yanjie Li,et al.  Rapid classification of Chinese quince (Chaenomeles speciosa Nakai) fruit provenance by near-infrared spectroscopy and multivariate calibration , 2016, Analytical and Bioanalytical Chemistry.

[16]  Tristan Perez,et al.  DeepFruits: A Fruit Detection System Using Deep Neural Networks , 2016, Sensors.

[17]  Giorgia Foca,et al.  Handling large datasets of hyperspectral images: reducing data size without loss of useful information. , 2013, Analytica chimica acta.

[18]  Kang Tu,et al.  Predict Compositions and Mechanical Properties of Sugar Beet Using Hyperspectral Scattering , 2016, Food and Bioprocess Technology.

[19]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[20]  Priya Antony,et al.  Antioxidant and anticancer properties of berries , 2018, Critical reviews in food science and nutrition.

[21]  Baohua Zhang,et al.  Principles, developments and applications of computer vision for external quality inspection of fruits and vegetables: A review , 2014 .

[22]  Meng-Han Hu,et al.  Classification and characterization of blueberry mechanical damage with time evolution using reflectance, transmittance and interactance imaging spectroscopy , 2016, Comput. Electron. Agric..

[23]  Liang Gong,et al.  Computer vision detection of defective apples using automatic lightness correction and weighted RVM classifier , 2015 .

[24]  Umezuruike Linus Opara,et al.  Estimating blueberry mechanical properties based on random frog selected hyperspectral data , 2015 .

[25]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[26]  Renfu Lu,et al.  Prediction of firmness and soluble solids content of blueberries using hyperspectral reflectance imaging , 2013 .

[27]  Chunjiang Zhao,et al.  Intelligent alerting for fruit-melon lesion image based on momentum deep learning , 2016, Multimedia Tools and Applications.

[28]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  M. W Gardner,et al.  Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences , 1998 .

[30]  Stane Krasovec The Future of Agriculture , 1929, Nature.

[31]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .