Identification of olive fruit, in intensive olive orchards, by means of its morphological structure using convolutional neural networks

Abstract Accurate yield estimation is a greatly desired objective in oliviculture due to the high economic value of its production. This paper presents a methodology aimed at achieving that end. It comprises an artificial-vision algorithm able to detect visible olives in digital images of olive trees captured directly in the field, at night-time and with artificial illumination. These images were taken in an intensive olive orchard of the Picual Olea europaea L. variety in September 2018 (two months prior to harvesting). Regarding the methodology, first, the images are pre-processed to generate a set of sub-images with high probability of containing an olive, thus reducing the search space by a magnitude of 103. Next, these sub-images are classified by a convolutional neural network (CNN) as olive, if they are centred in an olive fruit, or as other in any other case (even if they contain peripheral fruits). To train and validate the CNN, a special database called OLIVEnet was compiled with two disjoint sets integrating these sub-images. A training and a validation set was built with 234,168 and 299,946 olive and other sub-images, respectively. Five different CNN topologies were tested, correctly classifying the best performing one in 83.13% of olive instances, with a precision of 84.80%, and 99.12% of other instances; measured accuracy and F1 Score were 0.9822 and 0.8396, respectively. As far as the authors' knowledge goes, this article presents the first image analysis approach to automatically identify olive fruits in an image of the entire tree directly taken in the field. The obtained results constitute a first and solid step towards the implementation of an automatic system for yield estimation of olive orchards.