Automatic Pearl Classification Machine Based on a Multistream Convolutional Neural Network

In this paper, we design an automatic pearl classification machine, composed of four parts: feeding mechanism, delivering mechanism, vision-based detection device, and classification mechanism. Pearls can be delivered to the detection device one by one, where multiview images of each pearl can be collected. A novel multistream convolutional neural network (MS-CNN) is developed to cope with these multiview images, with each stream processing an image of particular viewing angle and different streams sharing part of weights to fuse high-order features without losing too much diversity. Using the machine, we collect 52 500 multiview images for 10 500 pearls, i.e., each pearl has five images of top, left, right, main, and rear views. These pearls were labeled by the experienced professionals in advance, and grouped into two classes with rough rules and seven classes with fine rules. Experimental results show that, compared with the support vector machine and backpropagation neural network, our MS-CNN behaves much better in both classification tasks, obtaining 92.14% and 91.24% accuracies. Moreover, the visualization of activations of convolutional kernels suggests that MS-CNN, imitating the manual process, can indeed recognize relatively complex features. These results indicate the potential value of our machine in the pearl industry.