Cloud-Based Monitoring of Thermal Anomalies in Industrial Environments Using AI and the Internet of Robotic Things

Recent advancements in cloud computing, artificial intelligence, and the internet of things (IoT) create new opportunities for autonomous industrial environments monitoring. Nevertheless, detecting anomalies in harsh industrial settings remains challenging. This paper proposes an edge-fog-cloud architecture with mobile IoT edge nodes carried on autonomous robots for thermal anomalies detection in aluminum factories. We use companion drones as fog nodes to deliver first response services and a cloud back-end for thermal anomalies analysis. We also propose a self-driving deep learning architecture and a thermal anomalies detection and visualization algorithm. Our results show our robot surveyors are low-cost, deliver reduced response time, and more accurately detect anomalies compared to human surveyors or fixed IoT nodes monitoring the same industrial area. Our self-driving architecture has a root mean square error of 0.19 comparable to VGG-19 with a significantly reduced complexity and three times the frame rate at 60 frames per second. Our thermal to visual registration algorithm maximizes mutual information in the image-gradient domain while adapting to different resolutions and camera frame rates.

[1]  Muhammed Oğuz TAŞ,et al.  Updating HD-Maps for Autonomous Transfer Vehicles in Smart Factories , 2018, 2018 6th International Conference on Control Engineering & Information Technology (CEIT).

[2]  Shen Yin,et al.  A Data-Driven Learning Approach for Nonlinear Process Monitoring Based on Available Sensing Measurements , 2017, IEEE Transactions on Industrial Electronics.

[3]  Sung Wook Baik,et al.  Activity Recognition Using Temporal Optical Flow Convolutional Features and Multilayer LSTM , 2019, IEEE Transactions on Industrial Electronics.

[4]  Mohamed Slim Masmoudi,et al.  Neural controller of autonomous driving mobile robot by an embedded camera , 2018, 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP).

[5]  Luigi Cinque,et al.  A multipurpose autonomous robot for target recognition in unknown environments , 2016, 2016 IEEE 14th International Conference on Industrial Informatics (INDIN).

[6]  Watalu Yamamoto,et al.  Anomaly Detection from Online Monitoring of System Operations Using Recurrent Neural Network , 2019, Procedia Manufacturing.

[7]  Xavier Savatier,et al.  The VIKINGS Autonomous Inspection Robot: Competing in the ARGOS Challenge , 2019, IEEE Robotics & Automation Magazine.

[8]  Tim Lüth,et al.  From autonomous assembly robots to service robots for factories , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[9]  George A. Bekey,et al.  On autonomous robots , 1998, The Knowledge Engineering Review.

[10]  A. A. Nippun Kumaar,et al.  QR Code based Path Planning for Warehouse Management Robot , 2018, 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[11]  Federico Pittino,et al.  Automatic Anomaly Detection on In-Production Manufacturing Machines Using Statistical Learning Methods , 2020, Sensors.

[12]  Shen Yin,et al.  Real-Time Monitoring and Control of Industrial Cyberphysical Systems: With Integrated Plant-Wide Monitoring and Control Framework , 2019, IEEE Industrial Electronics Magazine.

[13]  D. Ortenzi,et al.  Cooperative Robots Architecture for an Assistive Scenario , 2018, 2018 Zooming Innovation in Consumer Technologies Conference (ZINC).

[14]  Jae-Young Pyun,et al.  A Smartphone Indoor Positioning System Using Hybrid Localization Technology , 2019 .

[15]  Zhou Dong,et al.  A Survey on Anomaly Detection, Life Prediction and Maintenance Decision for Industrial Processes , 2013 .

[16]  Pradeep K. W. Abeygunawardhana,et al.  Design and Implementation of an Autonomous Indoor Surveillance Robot based on Raspberry Pi , 2019, 2019 International Conference on Advancements in Computing (ICAC).

[17]  Brent R. Young,et al.  Aluminium Process Fault Detection and Diagnosis , 2015 .

[18]  Lee Chia Chie,et al.  Artificial Landmark-based Indoor Navigation System for an Autonomous Unmanned Aerial Vehicle , 2020, 2020 IEEE 7th International Conference on Industrial Engineering and Applications (ICIEA).

[19]  Paulo Leitão,et al.  WsBot: A Tiny, Low-Cost Swarm Robot for Experimentation on Industry 4.0 , 2019, 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE).

[20]  Sung Wook Baik,et al.  CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks , 2020, Multimedia Tools and Applications.

[21]  Burkhard Hoppenstedt,et al.  Anomaly Detections for Manufacturing Systems Based on Sensor Data—Insights into Two Challenging Real-World Production Settings , 2019, Sensors.

[22]  Luis Montano.  Robots in challenging environments , 2019, 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[23]  Martin Styner,et al.  Parametric estimate of intensity inhomogeneities applied to MRI , 2000, IEEE Transactions on Medical Imaging.

[24]  Meer Shadman Saeed,et al.  Design and Implementation of a Dual Mode Autonomous Gas Leakage Detecting Robot , 2019, 2019 International Conference on Robotics,Electrical and Signal Processing Techniques (ICREST).

[25]  Rafael Rey,et al.  Human-robot co-working system for warehouse automation , 2019, 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[26]  Akshay Mogaveera,et al.  Self Driving Robot using Neural Network , 2018, 2018 International Conference on Information , Communication, Engineering and Technology (ICICET).

[27]  Hiroshi Yamamoto,et al.  Vehicle/Pedestrian Localization System Using Multiple Radio Beacons and Machine Learning for Smart Parking , 2019, 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[28]  Xiao-Sheng Si,et al.  A Survey on Anomaly Detection, Life Prediction and Maintenance Decision for Industrial Processes: A Survey on Anomaly Detection, Life Prediction and Maintenance Decision for Industrial Processes , 2014 .

[29]  Oliver Niggemann,et al.  System modeling based on machine learning for anomaly detection and predictive maintenance in industrial plants , 2014, Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA).