Deep multi-sensorial data analysis for production monitoring in hard metal industry

The industry practice of machining hard metal parts using CNC lathe turning machines is through grinding and milling procedures. The typical practice for quality control is through manual inspection, as automated solutions are difficult to integrate in production and do not reach the same level of accuracy. In this scope, the proposed system aims to automate the manufacturing process for the machine condition monitoring and 3D inspection of defective hard metal parts, by utilizing deep neural networks (DNNs) and investigating the defects on real production samples. Concretely, data are collected with (a) shop floor sensors, (b) high-resolution laser microprofilometer and (c) ultrasound scanner. The proposed system analyzes the collected data through AI models for quality control. Moreover, a fusion scheme is proposed to further improve accuracy. The system is validated on the classification of defective and non-defective samples, using metrics including accuracy, F-score, precision and recall for the performance evaluation.

[1]  Anil K. Jain,et al.  A system for 3D CAD-based inspection using range images , 1995, Pattern Recognit..

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  Dimitrios Tzovaras,et al.  A framework for inspection of dies attachment on PCB utilizing machine learning techniques , 2018 .

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Corina S. Pasareanu,et al.  DeepSafe: A Data-Driven Approach for Assessing Robustness of Neural Networks , 2018, ATVA.

[6]  Fei Shen,et al.  Machine Health Monitoring Using Local Feature-Based Gated Recurrent Unit Networks , 2018, IEEE Transactions on Industrial Electronics.

[7]  Michael Lütjen,et al.  Anomaly detection with convolutional neural networks for industrial surface inspection , 2019, Procedia CIRP.

[8]  Mian Li,et al.  Automated surface inspection for steel products using computer vision approach. , 2017, Applied optics.

[9]  Bo Yang,et al.  Machine learning-based evolution model and the simulation of a profit model of agricultural products logistics financing , 2019, Neural Computing and Applications.

[10]  Wolfgang Förstner,et al.  Plane Detection in Point Cloud Data , 2010 .

[11]  D. Tzovaras,et al.  Using Activity-Related Behavioural Features towards More Effective Automatic Stress Detection , 2012, PloS one.

[12]  Jun Zhu,et al.  Analyzing the Noise Robustness of Deep Neural Networks , 2018, 2018 IEEE Conference on Visual Analytics Science and Technology (VAST).

[13]  Mianxiong Dong,et al.  Deep Learning for Smart Industry: Efficient Manufacture Inspection System With Fog Computing , 2018, IEEE Transactions on Industrial Informatics.

[14]  Paul Checchin,et al.  Multi sensorial data fusion for efficient detection and tracking of road obstacles for inter-distance and anti-colision safety management , 2017, 2017 3rd International Conference on Control, Automation and Robotics (ICCAR).

[15]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[16]  Corina S. Pasareanu,et al.  DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks , 2017, ArXiv.

[17]  George Chryssolouris,et al.  Tool wear predictability estimation in milling based on multi-sensorial data , 2016 .

[18]  Fazel Ansari,et al.  Rethinking Human-Machine Learning in Industry 4.0: How Does the Paradigm Shift Treat the Role of Human Learning? , 2018 .

[19]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[20]  Maurice Pillet,et al.  The visual inspection of product surfaces , 2013 .

[21]  Jiong Tang,et al.  Preprocessing-Free Gear Fault Diagnosis Using Small Datasets With Deep Convolutional Neural Network-Based Transfer Learning , 2017, IEEE Access.

[22]  Shekhar Bhansali,et al.  Review—Deep Learning Methods for Sensor Based Predictive Maintenance and Future Perspectives for Electrochemical Sensors , 2020 .

[23]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[24]  Andres Bustillo,et al.  Smart optimization of a friction-drilling process based on boosting ensembles , 2018, Journal of Manufacturing Systems.

[25]  Paolo Napoletano,et al.  Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity , 2018, Sensors.

[26]  Hao Deng,et al.  A Generic Automated Surface Defect Detection Based on a Bilinear Model , 2019, Applied Sciences.

[27]  William John Teahan,et al.  Text classification and segmentation using minimum cross-entropy , 2000, RIAO.

[28]  Swagath Venkataramani,et al.  PACT: Parameterized Clipping Activation for Quantized Neural Networks , 2018, ArXiv.

[29]  Peng Wang,et al.  An Adaptive Multi-Sensor Data Fusion Method Based on Deep Convolutional Neural Networks for Fault Diagnosis of Planetary Gearbox , 2017, Sensors.

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  V. Hatzitaki,et al.  Application of intermittent galvanic vestibular stimulation reveals age-related constraints in the multisensory reweighting of posture , 2014, Neuroscience Letters.

[32]  Sergio Saludes Rodil,et al.  Monitoring and Fault Diagnosis in Manufacturing Processes in the Automotive Industry , 2011 .

[33]  S. S. Iyengar,et al.  A Low Cost Multi Sensorial Data Fusion for High Speed Obstacle Avoidance Using 3-D Point Clouds and Image Processing in Self Balancing Robots , 2017, 2017 2nd International Conference On Emerging Computation and Information Technologies (ICECIT).

[34]  Shahin Hedayati Kia,et al.  Information Fusion and Semi-Supervised Deep Learning Scheme for Diagnosing Gear Faults in Induction Machine Systems , 2019, IEEE Transactions on Industrial Electronics.

[35]  Ilya Safro,et al.  Machine Learning in Transportation Data Analytics , 2017 .

[36]  Rifat Kurban,et al.  Plane Segmentation of Kinect Point Clouds using RANSAC , 2015, ICIT 2015.

[37]  Bankole I. Oladapo,et al.  RETRACTED ARTICLE:3D printing of surface characterisation and finite element analysis improvement of PEEK-HAP-GO in bone implant , 2019, The International Journal of Advanced Manufacturing Technology.

[38]  Dimitrios Tzovaras,et al.  NEMESYS: Enhanced Network Security for Seamless Service Provisioning in the Smart Mobile Ecosystem , 2013, ISCIS.

[39]  Alessandro Massaro,et al.  Advanced Process Defect Monitoring Model and Prediction Improvement by Artificial Neural Network in Kitchen Manufacturing Industry: a Case of Study , 2019, 2019 II Workshop on Metrology for Industry 4.0 and IoT (MetroInd4.0&IoT).

[40]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[41]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[42]  Dimitrios Tzovaras,et al.  Fault Diagnosis in Microelectronics Attachment Via Deep Learning Analysis of 3-D Laser Scans , 2020, IEEE Transactions on Industrial Electronics.

[43]  Dawei Li,et al.  Automatic Detection and Classification of Sewer Defects via Hierarchical Deep Learning , 2019, IEEE Transactions on Automation Science and Engineering.

[44]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[45]  Michael Pedrick,et al.  Evaluation of ultrasonic inspection and imaging systems for concrete pipes , 2012 .

[46]  Michael G. Strintzis,et al.  3-D Model Search and Retrieval From Range Images Using Salient Features , 2010, IEEE Transactions on Multimedia.

[47]  Maozhen Li,et al.  Zero Defect Manufacturing of Microsemiconductors – An Application of Machine Learning and Artificial Intelligence , 2018, 2018 5th International Conference on Systems and Informatics (ICSAI).

[48]  Rahat Iqbal,et al.  Fault Detection and Isolation in Industrial Processes Using Deep Learning Approaches , 2019, IEEE Transactions on Industrial Informatics.

[49]  Yanqing Han,et al.  A structured light vision sensor for on-line weld bead measurement and weld quality inspection , 2019, The International Journal of Advanced Manufacturing Technology.

[50]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[51]  Mark S. Nixon,et al.  Acoustic information fusion for ground vehicle classification , 2008, 2008 11th International Conference on Information Fusion.

[52]  Diego Borro,et al.  A machine vision algorithm for quality control inspection of gears , 2020 .

[53]  S. Satorres Martínez,et al.  Quality inspection of machined metal parts using an image fusion technique , 2017 .

[54]  Dimitrios Tzovaras,et al.  A Deep Learning framework for simulation and defect prediction applied in microelectronics , 2020, Simul. Model. Pract. Theory.

[55]  Chris Yakopcic,et al.  A State-of-the-Art Survey on Deep Learning Theory and Architectures , 2019, Electronics.

[56]  Wei-Tsung Su,et al.  A Labor-Efficient GAN-based Model Generation Scheme for Deep-Learning Defect Inspection among Dense Beans in Coffee Industry , 2019, 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE).

[57]  Dazhong Wu,et al.  Deep learning for smart manufacturing: Methods and applications , 2018, Journal of Manufacturing Systems.

[58]  Jun Hu,et al.  Fault Location in Power Distribution Systems via Deep Graph Convolutional Networks , 2020, IEEE Journal on Selected Areas in Communications.

[59]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[60]  D. Hsu,et al.  Nondestructive testing using air-borne ultrasound. , 2006, Ultrasonics.

[61]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[62]  Satoshi Matsuoka,et al.  Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Dimitrios Tzovaras,et al.  Managing Spatial Graph Dependencies in Large Volumes of Traffic Data for Travel-Time Prediction , 2016, IEEE Transactions on Intelligent Transportation Systems.

[64]  John W. Sutherland,et al.  Predictive Maintenance of Machine Tool Systems Using Artificial Intelligence Techniques Applied to Machine Condition Data , 2019, Procedia CIRP.

[65]  Nawaz Mohamudally,et al.  Building An Anomaly Detection Engine (ADE) For IoT Smart Applications , 2018, FNC/MobiSPC.