Kiwifruit detection in field images using Faster R-CNN with VGG16

Abstract Kiwifruit is widely planted in Shaanxi, China, accounting for approximately 70% of the local production, and 33% of the global. Harvesting kiwifruits in China relies mainly on manual picking, and it is labor-intensive. To develop a machine vision system for harvesting robot which can work all day, kiwifruit images were captured in an orchard at different timing, morning, afternoon, and night, with or without flash, respectively. Kiwifruit images of 2400 were divided into training (1440) and testing (960) groups. A Faster R-CNN model implemented by VGG16 were constructed and trained. The average precision of VGG16 model was 87.61%, and the kiwifruit images collected under different timing and lighting conditions were detected well. In the end, the performance of the proposed method was compared with ZFNet in the same image dataset. It suggested that the proposed method achieved higher detection average precision than ZFNet (72.50%). This system is able to detect different categories of fruit in the field effectively and provides strong support for the harvesting robot, which can work all day round during the busy season.