Cultivar discrimination of litchi fruit images using deep learning

Abstract Litchi (Litchi chinensis Sonn.) originated from China and many of its cultivars have been produced in China so far during the long history of cultivation. One problem in litchi production and research is the worldwide confusion regarding litchi cultivar nomenclature. Because litchi cultivars can be described in terms of cultivar-dependent fruit appearance, it should be possible to discriminate cultivars of postharvest fruits. In this study, we explored this possibility using recently developed deep learning technology for four common Taiwanese cultivars 'Gui Wei', 'Hei Ye', 'No Mai Tsz', and 'Yu Her Pau'. First, we quantitatively evaluated litchi fruit shapes using elliptic Fourier descriptors and characterized the relationship between cultivars and fruit shapes. Results suggest that 'Yu Her Pau' can be clearly discriminated from others mainly based on its higher length-to-diameter ratio. We then fine-tuned a pre-trained VGG16 to construct a cultivar discrimination model. Relatively few images were sufficient to train the model to classify fruit images with 98.33% accuracy. We evaluated our model using images of fruits collected in different seasons and locations and found the model could identify 'Yu Her Pau' fruits with 100% accuracy and 'Hei Ye' fruits with 84% accuracy. A Grad-CAM visualization reveals that this model uses different cultivar-dependent regions for cultivar recognition. Overall, this study suggests that deep learning can be used to discriminate litchi cultivars from images of the fruit.

[1]  Charles R. Giardina,et al.  Elliptic Fourier features of a closed contour , 1982, Comput. Graph. Image Process..

[2]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[3]  Herbert Freeman,et al.  Computer Processing of Line-Drawing Images , 1974, CSUR.

[4]  S. Mitra,et al.  Origin, history, production and processing. , 2005 .

[5]  W. S. Rasband,et al.  ImageJ: Image processing and analysis in Java , 2012 .

[6]  X. Xiang,et al.  Developing a core collection of litchi (Litchi chinensis Sonn.) based on EST-SSR genotype data and agronomic traits , 2012 .

[7]  The Lychee Biotechnology , 2017 .

[8]  Sofia Visa,et al.  Modeling of tomato fruits into nine shape categories using elliptic fourier shape modeling and Bayesian classification of contour morphometric data , 2014, Euphytica.

[9]  Jer-Chia Chang,et al.  Litchi production and improvement in Taiwan. , 2009 .

[10]  F. Zee,et al.  Isozyme variation in lychee (Litchi chinensis Sonn.) , 1995 .

[11]  P. K. Pathak,et al.  Litchi production in the Asia-Pacific region. , 2010 .

[12]  D. Garrick,et al.  Quantitative evaluation of apple (Malus × domestica Borkh.) fruit shape by principal component analysis of Fourier descriptors , 2000, Euphytica.

[13]  H. Iwata,et al.  SHAPE: a computer program package for quantitative evaluation of biological shapes based on elliptic Fourier descriptors. , 2002, The Journal of heredity.

[14]  I. Ahmad,et al.  Genetic Diversity in Different Morphological Characteristics of Litchi (Litchi chinensis Sonn.) , 2004 .

[15]  B. Koul,et al.  Lychee Biology and Biotechnology , 2017, The Lychee Biotechnology.

[16]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[17]  Federico Pallottino,et al.  Quantitative evaluation of Tarocco sweet orange fruit shape using optoelectronic elliptic Fourier based analysis , 2009 .

[18]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[19]  Yudong Zhang,et al.  Image-based deep learning automated sorting of date fruit , 2019, Postharvest Biology and Technology.

[20]  M. A. Viruel,et al.  Development, characterization and variability analysis of microsatellites in lychee (Litchi chinensis Sonn., Sapindaceae) , 2004, Theoretical and Applied Genetics.

[21]  Guijun Yang,et al.  A rapid, low-cost deep learning system to classify squid species and evaluate freshness based on digital images , 2020 .

[22]  Takashi Akagi,et al.  Quantitative characterization of fruit shape and its differentiation pattern in diverse persimmon ( Diospyros kaki ) cultivars , 2018 .

[23]  S. Anuntalabhochai,et al.  Genetic diversity within Lychee (Litchi chinensis Sonn.) based on RAPD analysis , 2002 .

[24]  Po-An Chen,et al.  Litchi Breeding and Plant Management in Taiwan , 2017 .

[25]  Li Chengming,et al.  Cultivars and plant improvement. , 2005 .

[26]  Shan Lu.  Proceedings of the 8th Workshop on Programming Languages and Operating Systems , 2015, PLOS@SOSP.

[27]  Zihao Liu,et al.  Soft-shell Shrimp Recognition Based on an Improved AlexNet for Quality Evaluations , 2020 .

[28]  Fingerprinting and analysis of genetic diversity of litchi (Litchi chinensis Sonn.) accessions from different germplasm collections using microsatellite markers , 2013, Tree Genetics & Genomes.