Computer vision recognition of stem and calyx in apples using near-infrared linear-array structured light and 3D reconstruction

Abstract Automatic detection of common defects on apples by computer vision is still a challenge due to the similarity in appearance between true defects and stems/calyxes. Because the stem and calyx present a concave feature in apples, this paper proposes a novel stem and calyx recognition method using a computer vision system combined with near-infrared linear-array structured lighting and 3D reconstruction techniques to reveal this concavity. The 3D surface of the upper half of the inspected apples could be reconstructed by using a single multi-spectral camera and near-infrared linear-array structured light line by line on an adjustable speed conveyor belt. The height information for each pixel could be calculated by triangulation. Stems and calyxes would present a lower height than that of their neighbouring regions due to the local concave surface. In order to recognise the stems and calyxes efficiently, a standard spherical model (without stems and calyxes) is also constructed automatically, adapted to the size and boundary shape of the inspected apple. The difference between the 3D surface reconstruction and standard spherical model provides great potential for the recognition of stems and calyxes in apples. The final stem and calyx recognition algorithm was developed on the ratio images between 3D surface reconstruction images and standard spherical model construction images in gray level. The result had 97.5% overall recognition accuracy for the 100 samples (200 images), indicating that the proposed system and methods could be used for stem and calyx recognition.