Real-time hyperspectral imaging for the in-field estimation of strawberry ripeness with deep learning

Abstract Strawberry is one of the popular fruits with numerous nutrients. The ripeness of this fruits was estimated using the hyperspectral imaging (HSI) system in field and laboratory conditions in this study. Strawberry at early ripe and ripe stages were collected HSI data, covered wavelength ranges from 370 to 1015 nm. Spectral feature wavelengths were selected using the sequential feature selection (SFS) algorithm. Two wavelengths selected for field (530 and 604 nm) and laboratory (528 and 715 nm) samples, respectively. Then, reliability of such spectral features was validated based on support vector machine (SVM) classifier. Performance of SVM classification models had good results with receiver operating characteristic values for samples under both field and laboratory conditions higher than 0.95. Meanwhile, the spatial feature images were extracted from the spectral feature wavelength and the first three principal components for laboratory samples. Pretrained AlexNet convolutional neural network (CNN) was used to classify the early ripe and ripe strawberry samples, which obtained the accuracy of 98.6% for test dataset. The above results indicated real-time HSI system was promising for estimating strawberry ripeness under field and laboratory conditions, which could be a potential application technique for evaluating the harvesting time management for farmers and producers.

[1]  Samia Boukir,et al.  Support Vectors Selection for Supervised Learning Using an Ensemble Approach , 2010, 2010 20th International Conference on Pattern Recognition.

[2]  Dino Isa,et al.  Palm Oil Fresh Fruit Bunch Ripeness Grading Recognition Using Convolutional Neural Network , 2018 .

[3]  Xuan Liu,et al.  Automatic Detection of Single Ripe Tomato on Plant Combining Faster R-CNN and Intuitionistic Fuzzy Set , 2019, IEEE Access.

[4]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[5]  M. Ayora-Cañada,et al.  Hyperspectral FTIR imaging of olive fruit for understanding ripening processes , 2018, Postharvest Biology and Technology.

[6]  Olena Kravchuk,et al.  Sensory analysis of individual strawberry fruit and comparison with instrumental analysis , 2009 .

[7]  José M. Barat,et al.  Extending and Measuring the Quality of Fresh-cut Fruit and Vegetables: a Review , 2007 .

[8]  Di Wu,et al.  Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review — Part II: Applications , 2013 .

[9]  R. GeethaRamani,et al.  Identification of plant leaf diseases using a nine-layer deep convolutional neural network , 2019, Comput. Electr. Eng..

[10]  A. Peirs,et al.  Nondestructive measurement of fruit and vegetable quality by means of NIR spectroscopy: A review , 2007 .

[11]  En Li,et al.  Apple detection during different growth stages in orchards using the improved YOLO-V3 model , 2019, Comput. Electron. Agric..

[12]  Fengfeng Zhang,et al.  A deep learning based feature extraction method on hyperspectral images for nondestructive prediction of TVB-N content in Pacific white shrimp (Litopenaeus vannamei) , 2019, Biosystems Engineering.

[13]  Zhenglin Wang,et al.  Mango Fruit Load Estimation Using a Video Based MangoYOLO—Kalman Filter—Hungarian Algorithm Method , 2019, Sensors.

[14]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[15]  Lav R. Khot,et al.  Optical sensing for early spring freeze related blueberry bud damage detection: Hyperspectral imaging for salient spectral wavelengths identification , 2019, Comput. Electron. Agric..

[16]  Yudong Zhang,et al.  A Rule-Based Model for Bankruptcy Prediction Based on an Improved Genetic Ant Colony Algorithm , 2013 .

[17]  P. From,et al.  Instance Segmentation and Localization of Strawberries in Farm Conditions for Automatic Fruit Harvesting , 2019, IFAC-PapersOnLine.

[18]  Min Zuo,et al.  CropDeep: The Crop Vision Dataset for Deep-Learning-Based Classification and Detection in Precision Agriculture , 2019, Sensors.

[19]  Yang Yu,et al.  Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN , 2019, Comput. Electron. Agric..

[20]  Johannes R. Sveinsson,et al.  Spectral and Spatial Classification of Hyperspectral Data Using SVMs and Morphological Profiles , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Tristan Perez,et al.  Fruit Quantity and Ripeness Estimation Using a Robotic Vision System , 2018, IEEE Robotics and Automation Letters.

[22]  Tristan Perez,et al.  DeepFruits: A Fruit Detection System Using Deep Neural Networks , 2016, Sensors.

[23]  Dean Zhao,et al.  Cucumber Fruits Detection in Greenhouses Based on Instance Segmentation , 2019, IEEE Access.

[24]  Jing Zhang,et al.  Branch detection for apple trees trained in fruiting wall architecture using depth features and Regions-Convolutional Neural Network (R-CNN) , 2018, Comput. Electron. Agric..

[25]  Marcel Salathé,et al.  Using Deep Learning for Image-Based Plant Disease Detection , 2016, Front. Plant Sci..

[26]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[27]  Anwar P.P. Abdul Majeed,et al.  Autonomous tomato harvesting robotic system in greenhouses: deep learning classification , 2019 .

[28]  Ghulam Muhammad,et al.  Automatic Fruit Classification Using Deep Learning for Industrial Applications , 2019, IEEE Transactions on Industrial Informatics.

[29]  José Blasco,et al.  Non-destructive assessment of the internal quality of intact persimmon using colour and VIS/NIR hyperspectral imaging , 2017 .

[30]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[31]  Chyi-Yeu Lin,et al.  Deep learning for noninvasive classification of clustered horticultural crops – A case for banana fruit tiers , 2019, Postharvest Biology and Technology.

[32]  Songlin Zhuang,et al.  Detection and classification of bruises of pears based on thermal images , 2020 .

[33]  Chu Zhang,et al.  Hyperspectral imaging analysis for ripeness evaluation of strawberry with support vector machine , 2016 .

[34]  Konstantinos P. Ferentinos,et al.  Deep learning models for plant disease detection and diagnosis , 2018, Comput. Electron. Agric..

[35]  Raimund Leitner,et al.  Hyperspectral fruit and vegetable classification using convolutional neural networks , 2019, Comput. Electron. Agric..

[36]  Fei Liu,et al.  Ripeness Classification of Astringent Persimmon Using Hyperspectral Imaging Technique , 2014, Food and Bioprocess Technology.

[37]  Maryam Rahnemoonfar,et al.  Deep Count: Fruit Counting Based on Deep Simulated Learning , 2017, Sensors.

[38]  James Patrick Underwood,et al.  Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards , 2016, J. Field Robotics.

[39]  Guan Gui,et al.  Deep Learning Based Improved Classification System for Designing Tomato Harvesting Robot , 2018, IEEE Access.

[40]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Di Wu,et al.  Development of deep learning method for predicting firmness and soluble solid content of postharvest Korla fragrant pear using Vis/NIR hyperspectral reflectance imaging , 2018 .

[42]  Zaaba Ahmad,et al.  Automatic Classification of Mangosteen Ripening Stages using Deep Learning , 2019, 2019 1st International Conference on Artificial Intelligence and Data Sciences (AiDAS).

[43]  Yuanyuan Shao,et al.  Determination of the bruise degree for cherry using Vis-NIR reflection spectroscopy coupled with multivariate analysis , 2019, PloS one.

[44]  Ning Wang,et al.  Studies on banana fruit quality and maturity stages using hyperspectral imaging , 2012 .

[45]  Aoife Gowen,et al.  Ripeness Classification of Bananito Fruit (Musa acuminata,AA): a Comparison Study of Visible Spectroscopy and Hyperspectral Imaging , 2019, Food Analytical Methods.

[46]  José Manuel Amigo,et al.  Ripeness monitoring of two cultivars of nectarine using VIS-NIR hyperspectral reflectance imaging , 2017 .