Deep learning for the identification of bruised apples by fusing 3D deep features for apple grading systems

Abstract Automated bruised apple detection is an important application in the fruit industry. In this paper, we investigated convolutional neural network-based predictive models for the identification of bruised apples based on shape information (in a form of three-dimensional [3D] surface meshes) acquired from a 3D infrared imaging system. There are often irregularities on bruised apple surfaces, which can be used to differentiate those bruised apples from unbruised ones. In this study, we adopted transformation approaches so that geometric information located on 3D surface meshes could be efficiently converted to two-dimensional (2D) “image-like” feature maps. Our transformation approaches allowed current state-of-the-art deep learning models to be directly used without modifications. During algorithmic developments, three different configurations of deep convolutional neural networks were investigated in conjunction with transfer learning. We also explored different fusion strategies to further improve the performance of our predictive models. We found that the identification rate of our best predictive model was 97.67%, which markedly exceeded the best performance of a handcraft feature engineering method developed by our group. Our preliminary results show that the proposed method has great potential and further developments will lead to an automated system for apple bruise detection.