Detecting ripe fruits under natural occlusion and illumination conditions

Abstract For an accurate detection of ripe fruits under uneven illumination and ubiquitous occlusion conditions, this paper proposes a method to detect and locate ripe fruits based on machine vision. There are four key steps in this method including image graying and background removal, binary image optimization, true contour fragments extraction, and fruit fitting. For testing the proposed method, field experiments were conducted with tomato and citrus, and the ripe fruits in complex environments were successfully detected and located. From the detection experiments, it showed that the recognition rate for ripe fruits in the near zone of the proposed method was higher than 97.44%, the average time consumption was 0.2966 s, and the positioning error was less than 4.41%. In addition, it can be concluded from the comparative experiment that the proposed method is superior to conventional Hough transform, random Hough transform, and other methods based on deep learning in terms of detection rate, time performance and positioning accuracy. Therefore, it can be applied to picking robots for real-time detecting and locating ripe fruits.

[1]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[2]  Kamel Barkaoui,et al.  Performability evaluation of server virtualized systems under bursty workload , 2018 .

[3]  Xuewei Chao,et al.  Semi-supervised few-shot learning approach for plant diseases recognition , 2021, Plant Methods.

[4]  Hiroki Imamura,et al.  A Robust Recognition Method for Occlusion of Mini Tomatoes Based on Hue Information and the Curvature , 2015, Int. J. Image Graph..

[5]  G. van Straten,et al.  A vision based row detection system for sugar beet , 2005 .

[6]  Unai Irusta,et al.  Few-Shot Learning approach for plant disease classification using images taken in the field , 2020, Comput. Electron. Agric..

[7]  Seishi Ninomiya,et al.  On Plant Detection of Intact Tomato Fruits Using Image Analysis and Machine Learning Methods , 2014, Sensors.

[8]  Ahmad Al-Mallahi,et al.  A novel image processing algorithm to separate linearly clustered kiwifruits , 2019, Biosystems Engineering.

[9]  Xiaoyang Liu,et al.  A method of segmenting apples at night based on color and position information , 2016, Comput. Electron. Agric..

[10]  Pedro Javier Herrera,et al.  Robust digital control for autonomous skid-steered agricultural robots , 2018, Comput. Electron. Agric..

[11]  Tristan Perez,et al.  DeepFruits: A Fruit Detection System Using Deep Neural Networks , 2016, Sensors.

[12]  Liang Gong,et al.  Detecting tomatoes in greenhouse scenes by combining AdaBoost classifier and colour analysis , 2016 .

[13]  Guoxu Liu,et al.  YOLO-Tomato: A Robust Algorithm for Tomato Detection Based on YOLOv3 , 2020, Sensors.

[14]  Raphael Linker,et al.  Determination of the number of green apples in RGB images recorded in orchards , 2012 .

[15]  Hakil Kim,et al.  A critical review on computer vision and artificial intelligence in food industry , 2020, Journal of Agriculture and Food Research.

[16]  Yael Edan,et al.  Adaptive thresholding with fusion using a RGBD sensor for red sweet-pepper detection , 2016 .

[17]  Shuyi Mao,et al.  A Mature-Tomato Detection Algorithm Using Machine Learning and Color Analysis † , 2019, Sensors.

[18]  Richard I. Kitney,et al.  A direct method for least-squares circle fitting , 1991 .

[19]  Yung-Sheng Chen,et al.  A modified fast parallel algorithm for thinning digital patterns , 1988, Pattern Recognit. Lett..

[20]  Yang Li,et al.  Meta-learning baselines and database for few-shot classification in agriculture , 2021, Comput. Electron. Agric..

[21]  Victor Alchanatis,et al.  Image fusion of visible and thermal images for fruit detection. , 2009 .

[22]  Zhao Dean,et al.  Recognition of apple fruit in natural environment , 2016 .

[23]  Yuanshen Zhao,et al.  Robust Tomato Recognition for Robotic Harvesting Using Feature Images Fusion , 2016, Sensors.

[24]  Yibin Ying,et al.  Recognition of clustered tomatoes based on binocular stereo vision , 2014 .

[25]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[26]  En Li,et al.  Apple detection during different growth stages in orchards using the improved YOLO-V3 model , 2019, Comput. Electron. Agric..

[27]  Hossein Khosravi,et al.  A deep neural network approach towards real-time on-branch fruit recognition for precision horticulture , 2020, Expert Syst. Appl..

[28]  Rui Lin,et al.  Colored rice quality inspection system using machine vision , 2019, Journal of Cereal Science.

[29]  Guijun Yang,et al.  A proposed framework for accelerating technology trajectories in agriculture: a case study in China , 2018 .

[30]  Man Zhang,et al.  Mature Tomato Fruit Detection Algorithm Based on improved HSV and Watershed Algorithm , 2018 .

[31]  Gonzalo Pajares,et al.  Automatic detection of curved and straight crop rows from images in maize fields , 2017 .

[32]  Rui Li,et al.  Kiwifruit detection in field images using Faster R-CNN with VGG16 , 2019, IFAC-PapersOnLine.

[33]  Yang Li,et al.  Few-shot cotton pest recognition and terminal realization , 2020, Comput. Electron. Agric..

[34]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Denis Stajnko,et al.  Original papers: Detecting fruits in natural scenes by using spatial-frequency based texture analysis and multiview geometry , 2011 .

[36]  Jun Zhou,et al.  Automatic Recognition of Ripening Tomatoes by Combining Multi-Feature Fusion with a Bi-Layer Classification Strategy for Harvesting Robots , 2019, Sensors.

[37]  Zhen Liu,et al.  The recognition of litchi clusters and the calculation of picking point in a nocturnal natural environment , 2018 .

[38]  Wu-Chih Hu,et al.  A rotationally invariant two-phase scheme for corner detection , 1996, Pattern Recognit..

[39]  Won Suk Lee,et al.  Green citrus detection using 'eigenfruit', color and circular Gabor texture features under natural outdoor conditions , 2011 .

[40]  Juntao Xiong,et al.  A visual detection method for nighttime litchi fruits and fruiting stems , 2020, Comput. Electron. Agric..

[41]  Flavio B. P. Malavazi,et al.  LiDAR-only based navigation algorithm for an autonomous agricultural robot , 2018, Comput. Electron. Agric..

[42]  F. García-Luna,et al.  Towards an artificial vision-robotic system for tomato identification , 2016 .

[43]  Prasanna Rangarajan,et al.  Hyper least squares fitting of circles and ellipses , 2011, Comput. Stat. Data Anal..

[44]  Erkki Oja,et al.  A new curve detection method: Randomized Hough transform (RHT) , 1990, Pattern Recognit. Lett..

[45]  Xiangjun Zou,et al.  Banana detection based on color and texture features in the natural environment , 2019, Comput. Electron. Agric..

[46]  Mukesh Tripathi,et al.  A role of computer vision in fruits and vegetables among various horticulture products of agriculture fields: A survey , 2020 .