Deep Learning for Industrial AI: Challenges, New Methods and Best Practices

Industrial AI is concerned with the application of Artificial Intelligence (AI), Machine Learning (ML) and related technologies towards addressing real-world use cases in industrial and societal domains. These uses cases can be broadly categorized into the horizontal areas of maintenance and repair, operations and supply chain, quality, safety, design, and end-to-end optimization - with applications in a variety of verticals. In the last few years, we have witnessed a growing interest in applying Deep Learning (DL) techniques to Industrial AI problems, ranging from using sequence models such as Long Short-Term Memory (LSTM) for predicting failures in equipment, to using Deep Reinforcement Learning (Deep RL) for scheduling and dispatching. Applying deep learning techniques to industrial applications imposes a set of unique challenges, which include, but are not limited to, (1) limited data, highly skewed class distribution and occurrence of rare classes such as failures, (2) multi-modal data (sensors, events, images, text, etc.) indexed over space and time (3) the need for explainable decisions, (4) a need to attain consistency between different but "related" models and between multiple generations of the same model, and (5) decision making to optimize business outcomes where the cost of a mistake could be very high. This tutorial presents an overview of these challenges, along with new methods and best practices to address them. Examples of these methods include using sequence DL models and Functional Neural Networks (FNNs) for modeling sensor and spatiotemporal measurements; using multi-task learning, graph models and ensemble learning for improving consistency of DL models; using deep RL for health indicator learning and dynamic dispatching; cost-based decision making for prognostics; and using GANs for generating senor data for prognostics. Finally, we will present some open problems in Industrial AI and how the research community can shape the future of the next industrial and societal revolution.

[1]  Jure Skvarč,et al.  Segmentation-based deep-learning approach for surface-defect detection , 2019, Journal of Intelligent Manufacturing.

[2]  Susumu Serita,et al.  Multilayer Perceptron for Sparse Functional Data , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[3]  Chetan Gupta,et al.  Long Short-Term Memory Network for Remaining Useful Life estimation , 2017, 2017 IEEE International Conference on Prognostics and Health Management (ICPHM).

[4]  Chetan Gupta,et al.  Two Birds with One Network: Unifying Failure Event Prediction and Time-to-failure Modeling , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[5]  Stephan Spiegel,et al.  Cost-Sensitive Learning for Predictive Maintenance , 2018, ArXiv.

[6]  Peter Henderson,et al.  Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control , 2017, ArXiv.

[7]  Chetan Gupta,et al.  Remaining Useful Life Estimation for Systems with Abrupt Failures , 2018, Annual Conference of the PHM Society.

[8]  F. Rossi,et al.  Functional data analysis with multi layer perceptrons , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[9]  Chetan Gupta,et al.  Building chatbots from large scale domain-specific knowledge bases: challenges and opportunities , 2020, 2020 IEEE International Conference on Prognostics and Health Management (ICPHM).

[10]  Arman Hasanzadeh,et al.  Fault Detection and Isolation in Industrial Networks using Graph Convolutional Neural Networks , 2019, 2019 IEEE International Conference on Prognostics and Health Management (ICPHM).

[11]  Sreerupa Das,et al.  Automating Visual Inspection with Convolutional Neural Networks , 2019 .

[12]  Chetan Gupta,et al.  Equipment Health Indicator Learning Using Deep Reinforcement Learning , 2018, ECML/PKDD.

[13]  A. Chandramouli,et al.  Unsupervised Extraction of Part Names from Service Logs , 2022 .

[14]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[15]  Susumu Serita,et al.  Manufacturing Dispatching using Reinforcement and Transfer Learning , 2019, ECML/PKDD.

[16]  Chi Zhang,et al.  Risk-Based Dynamic Pricing via Failure Prediction , 2019, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA).

[17]  Chetan Gupta,et al.  Discriminant Generative Adversarial Networks with its Application to Equipment Health Classification , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Chetan Gupta,et al.  Remaining Useful Life Estimation Using Functional Data Analysis , 2019, 2019 IEEE International Conference on Prognostics and Health Management (ICPHM).

[19]  Ahmed K. Farahat,et al.  Generative Adversarial Networks for Failure Prediction , 2019, ECML/PKDD.

[20]  Lovekesh Vig,et al.  Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder , 2016, ArXiv.

[21]  Prasad Patil,et al.  Training replicable predictors in multiple studies , 2018, Proceedings of the National Academy of Sciences.

[22]  Chetan Gupta,et al.  Trace Norm Generative Adversarial Networks for Sensor Generation and Feature Extraction , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).