Auto-Sorting System Toward Smart Factory Based on Deep Learning for Image Segmentation

Machine part sorting is important and monotonous in smart factory. In this paper, an auto-sorting system is proposed based on the deep learning method. In the proposed system, an industrial objection detection network combined with a robotic arm controlling system is designed to automatically and efficiently complete machine part sorting. Region-based full convolutional network (R-FCN) is applied for locating and recognizing different types of images of industrial object models. After comparison and simulation analysis, it illustrated that the R-FCN model trained with enough labeled data can efficiently and accurately recognize the object from the images captured by visual sensors. Furthermore, with enough data, the network can be robust to view angle rotation both vertically and horizontally, and a small part of overlapping of object will not mislead the judgment of the network in most situations. The case study results illustrate that the position and type of objects can be successfully detected. The code will be available publicly at https://github.com/tianwangbuaa/.

[1]  Yan Yang,et al.  A STEP-based machining data model for autonomous process generation of intelligent CNC controller , 2018 .

[2]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Amit Kumar Singh,et al.  Automatic sorting of object by their colour and dimension with speed or process control of induction motor , 2017, 2017 International Conference on Circuit ,Power and Computing Technologies (ICCPCT).

[4]  Anna Syberfeldt,et al.  Localizing operators in the smart factory: A review of existing techniques and systems , 2016, 2016 International Symposium on Flexible Automation (ISFA).

[5]  Ferdinando Cannella,et al.  An automatic assembly parts detection and grasping system for industrial manufacturing , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[6]  Fei Tao,et al.  IoT-Based Intelligent Perception and Access of Manufacturing Resource Toward Cloud Manufacturing , 2014, IEEE Transactions on Industrial Informatics.

[7]  Itziar Cabanes,et al.  An Autonomous Transport Vehicle in an existing manufacturing facility with focus on the docking maneuver task , 2017, 2017 3rd International Conference on Control, Automation and Robotics (ICCAR).

[8]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[9]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[10]  Andrew Kusiak,et al.  Data-driven smart manufacturing , 2018, Journal of Manufacturing Systems.

[11]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[12]  Fei Tao,et al.  CCIoT-CMfg: Cloud Computing and Internet of Things-Based Cloud Manufacturing Service System , 2014, IEEE Transactions on Industrial Informatics.

[13]  Ying Liu,et al.  Agent and Cyber-Physical System Based Self-Organizing and Self-Adaptive Intelligent Shopfloor , 2017, IEEE Transactions on Industrial Informatics.

[14]  John M. Prager,et al.  Extracting and Labeling Boundary Segments in Natural Scenes , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Sarp Ertürk,et al.  Automatic waste sorting using shortwave infrared hyperspectral imaging system , 2013, 2013 5th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[18]  Tianliang Hu,et al.  Design and development of a CNC machining process knowledge base using cloud technology , 2016, The International Journal of Advanced Manufacturing Technology.

[19]  Hichem Snoussi,et al.  Internal Transfer Learning for Improving Performance in Human Action Recognition for Small Datasets , 2017, IEEE Access.

[20]  Hichem Snoussi,et al.  Abnormal event detection via covariance matrix for optical flow based feature , 2018, Multimedia Tools and Applications.

[21]  Kevin Ashton,et al.  That ‘Internet of Things’ Thing , 1999 .

[22]  H. Kagermann Change Through Digitization—Value Creation in the Age of Industry 4.0 , 2015 .

[23]  Yan Yang,et al.  A knowledge generation mechanism of machining process planning using cloud technology , 2019, J. Ambient Intell. Humaniz. Comput..

[24]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Hichem Snoussi,et al.  Detection of Abnormal Visual Events via Global Optical Flow Orientation Histogram , 2014, IEEE Transactions on Information Forensics and Security.

[26]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[27]  Wan-Jung Chang,et al.  Real-Time Monitoring via Patch-Type Piezoelectric Force Sensors for Internet of Things Based Logistics , 2017, IEEE Sensors Journal.

[28]  Nguyen Truong Thinh,et al.  Using ANFIS to predict picking position of the fruits sorting system , 2017, 2017 International Conference on System Science and Engineering (ICSSE).

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Fei Tao,et al.  SDMSim: A manufacturing service supply–demand matching simulator under cloud environment , 2017 .

[31]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Chris A. Glasbey,et al.  An Analysis of Histogram-Based Thresholding Algorithms , 1993, CVGIP Graph. Model. Image Process..

[33]  S. Hata,et al.  Improvement of recognition rate of towel mark recognition system , 2012, 2012 9th France-Japan & 7th Europe-Asia Congress on Mechatronics (MECATRONICS) / 13th Int'l Workshop on Research and Education in Mechatronics (REM).

[34]  Chai Xu-dong,et al.  Cloud manufacturing:a new service-oriented networked manufacturing model , 2010 .

[35]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[36]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[38]  Hichem Snoussi,et al.  A fast and robust convolutional neural network-based defect detection model in product quality control , 2017, The International Journal of Advanced Manufacturing Technology.

[39]  Ataollah Ebrahimzadeh,et al.  An efficient system for automatic sorting of the ceramic tiles , 2010, 6th International Conference on Digital Content, Multimedia Technology and its Applications.

[40]  Subhas Chandra Mukhopadhyay,et al.  Smart Sensors and Internet of Things: A Postgraduate Paper , 2017, IEEE Sensors Journal.

[41]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.