Super-High-Purity Seed Sorter Using Low-Latency Image-Recognition Based on Deep Learning

Most commercial optical sorting systems are designed to achieve high throughput, so they use a naive low-latency image processing for object identification. These naive low-latency algorithms have difficulty in accurately identifying objects with various shapes, textures, sizes, and colors, so the purity of sorted objects is degraded. Current deep learning technology enables robust image detection and classification, but its inference latency requires several milliseconds; thus, deep learning cannot be directly applied to such real-time high throughput applications. We therefore developed a super-high purity seed sorting system that uses a low-latency image-recognition based on a deep neural network and removes the seeds of noxious weeds from mixed seed product at high throughput with accuracy. The proposed system partitions the detection task into localization and classification, and applies batch inference only once strategy; it achieved 500-fps throughput image-recognition including detection and tracking. Based on the classified and tracked results, air ejectors expel the unwanted seeds. This proposed system eliminates almost the whole weeds with small loss of desired seeds, and is superior to current commercial optical sorting systems.

[1]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  M. C. Pasikatan,et al.  SORTING SYSTEMS BASED ON OPTICAL METHODS FOR DETECTING AND REMOVING SEEDS INFESTED INTERNALLY BY INSECTS OR FUNGI: A REVIEW* , 2001 .

[3]  Tom C. Pearson,et al.  Machine vision system for automated detection of stained pistachio nuts , 1995, Other Conferences.

[4]  David A. Hendrickson,et al.  Review of the Impact of Mechanical Harvesting and Optical Berry Sorting on Grape and Wine Composition , 2017, Catalyst: Discovery into Practice.

[5]  Tom C. Pearson,et al.  Hardware-based image processing for high-speed inspection of grains , 2009 .

[6]  A. Stallings Sunn hemp (Crotalaria juncea L.) as a Cover Crop for Winter Wheat , 2015 .

[7]  M. Roederer,et al.  The history and future of the fluorescence activated cell sorter and flow cytometry: a view from Stanford. , 2002, Clinical chemistry.

[8]  Stefan Roth,et al.  MOT16: A Benchmark for Multi-Object Tracking , 2016, ArXiv.

[9]  Bayram Cetisli,et al.  Design of an automatic apple sorting system using machine vision , 2016, Comput. Electron. Agric..

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  N. Funk A Study of the Kalman Filter applied to Visual Tracking , 2003 .

[12]  Ah Farsaie,et al.  Design and Development of an Automatic Electro-Optical Sorter for Removing BGY Fluorescent Pistachio Nuts , 1981 .

[13]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[14]  Boštjan Likar,et al.  Automated visual inspection of imprinted pharmaceutical tablets , 2007 .

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  H. Kuhn The Hungarian method for the assignment problem , 1955 .

[18]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[19]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  S. Hirano,et al.  Near Infra Red Detection of Internally Moldy Nuts. , 1998, Bioscience, biotechnology, and biochemistry.

[24]  Anup Vibhute,et al.  Applications of Image Processing in Agriculture: A Survey , 2012 .

[25]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Seokhoon Jeong,et al.  Development of an automatic sorting system for fresh ginsengs by image processing techniques , 2017, Human-centric Computing and Information Sciences.