Deep Reinforcement Learning-Based Online Domain Adaptation Method for Fault Diagnosis of Rotating Machinery

Deep-learning-based methods have been successfully applied to fault diagnosis of rotating machinery. However, the domain mismatch among different operating conditions significantly deteriorates diagnostic performance of these methods in industrial applications. To solve this problem, a new fault diagnosis model based on capsule neural network (Cap-net) is constructed, and a novel online domain adaptation learning method based on deep reinforcement learning (DRL) is proposed in this article to improve the adaptivity of the fault diagnosis model. In this method, the Cap-net is first introduced into the DRL as an agent to extract representative features and diagnoses fault. Moreover, the online domain adaptation learning of the agent is conducted based on the Q-learning of the DRL so as to adapt to different operating domains that have never been experienced. Specifically, an online feature dictionary combined with cosine similarity is designed to coarsely label the online data collected from different operating domain, while a reward mechanism is defined to evaluate the obtained label. Subsequently, the online data, the corresponding label, and the reward are used to optimize the agent to obtain the desired diagnostic model. Two experiment studies are implemented to verify the effectiveness of the proposed method. The experimental results show that the proposed method has more excellent diagnostic performance and adaptivity than the existing popular methods.

[1]  Xinyu Shao,et al.  Autoencoder Quasi-Recurrent Neural Networks for Remaining Useful Life Prediction of Engineering Systems , 2021, IEEE/ASME Transactions on Mechatronics.

[2]  Chao Deng,et al.  Convolutional Neural Network-Based Bayesian Gaussian Mixture for Intelligent Fault Diagnosis of Rotating Machinery , 2021, IEEE Transactions on Instrumentation and Measurement.

[3]  Xinyu Shao,et al.  Remaining Useful Life Prognosis Based on Ensemble Long Short-Term Memory Neural Network , 2021, IEEE Transactions on Instrumentation and Measurement.

[4]  Haidong Shao,et al.  Ensemble transfer CNNs driven by multi-channel signals for fault diagnosis of rotating machinery cross working conditions , 2020, Knowl. Based Syst..

[5]  Xinyu Shao,et al.  Ensemble Generalized Multiclass Support-Vector-Machine-Based Health Evaluation of Complex Degradation Systems , 2020, IEEE/ASME Transactions on Mechatronics.

[6]  Thomas Parisini,et al.  State of AI-Based Monitoring in Smart Manufacturing and Introduction to Focused Section , 2020, IEEE/ASME Transactions on Mechatronics.

[7]  M.M.A. Mahfouz,et al.  A protection scheme for multi-distributed smart microgrid based on auto-cosine similarity of feeders current patterns , 2020 .

[8]  Zheng Liu,et al.  Intelligent Fault Diagnosis of Multichannel Motor–Rotor System Based on Multimanifold Deep Extreme Learning Machine , 2020, IEEE/ASME Transactions on Mechatronics.

[9]  Albert Y. Zomaya,et al.  Reinforcement learning in sustainable energy and electric systems: a survey , 2020, Annu. Rev. Control..

[10]  Xinyu Shao,et al.  Reliability prediction of machinery with multiple degradation characteristics using double-Wiener process and Monte Carlo algorithm , 2019 .

[11]  Kun Jiang,et al.  A deep capsule neural network with stochastic delta rule for bearing fault diagnosis on raw vibration signals , 2019 .

[12]  Xiang Li,et al.  Cross-Domain Fault Diagnosis of Rolling Element Bearings Using Deep Generative Neural Networks , 2019, IEEE Transactions on Industrial Electronics.

[13]  Haoxiang Lang,et al.  Adaptive System Identification and Severity Index-Based Fault Diagnosis in Motors , 2019, IEEE/ASME Transactions on Mechatronics.

[14]  Cheng Wu,et al.  Domain Space Transfer Extreme Learning Machine for Domain Adaptation , 2019, IEEE Transactions on Cybernetics.

[15]  Chao Liu,et al.  An adaptive spatiotemporal feature learning approach for fault diagnosis in complex systems , 2019, Mechanical Systems and Signal Processing.

[16]  Jun Wu,et al.  Machine Health Monitoring Using Adaptive Kernel Spectral Clustering and Deep Long Short-Term Memory Recurrent Neural Networks , 2019, IEEE Transactions on Industrial Informatics.

[17]  Chao Liu,et al.  A novel adversarial learning framework in deep convolutional neural network for intelligent diagnosis of mechanical faults , 2019, Knowl. Based Syst..

[18]  Zhencai Zhu,et al.  A New Piezoelectric Bimorph Energy Harvester Based on the Vortex-Induced-Vibration Applied in Rotational Machinery , 2019, IEEE/ASME Transactions on Mechatronics.

[19]  Huijun Gao,et al.  A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis , 2019, Neurocomputing.

[20]  Robert X. Gao,et al.  Deep learning and its applications to machine health monitoring , 2019, Mechanical Systems and Signal Processing.

[21]  Liang Gao,et al.  A New Deep Transfer Learning Based on Sparse Auto-Encoder for Fault Diagnosis , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Chao Liu,et al.  Deep Transfer Network with Joint Distribution Adaptation: A New Intelligent Fault Diagnosis Framework for Industry Application , 2018, ISA transactions.

[23]  Mevludin Glavic,et al.  (Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectives , 2019, Annu. Rev. Control..

[24]  Min Xia,et al.  Fault Diagnosis for Rotating Machinery Using Multiple Sensors and Convolutional Neural Networks , 2018, IEEE/ASME Transactions on Mechatronics.

[25]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[26]  M. Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[27]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[28]  Chen Lu,et al.  Intelligent fault diagnosis of rolling bearing using hierarchical convolutional network based health state classification , 2017, Adv. Eng. Informatics.

[29]  Peng Wang,et al.  An Adaptive Multi-Sensor Data Fusion Method Based on Deep Convolutional Neural Networks for Fault Diagnosis of Planetary Gearbox , 2017, Sensors.

[30]  Moncef Gabbouj,et al.  Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks , 2016, IEEE Transactions on Industrial Electronics.

[31]  Cong Wang,et al.  Construction of hierarchical diagnosis network based on deep learning and its application in the fault pattern recognition of rolling element bearings , 2016 .

[32]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[33]  Walter Sextro,et al.  Condition Monitoring of Bearing Damage in Electromechanical Drive Systems by Using Motor Current Signals of Electric Motors: A Benchmark Data Set for Data-Driven Classification , 2016, PHM Society European Conference.

[34]  Haidong Shao,et al.  Rolling bearing fault diagnosis using an optimization deep belief network , 2015 .

[35]  Noureddine Zerhouni,et al.  Bearing Health Monitoring Based on Hilbert–Huang Transform, Support Vector Machine, and Regression , 2015, IEEE Transactions on Instrumentation and Measurement.

[36]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[37]  Ruoyu Li,et al.  Plastic Bearing Fault Diagnosis Based on a Two-Step Data Mining Approach , 2013, IEEE Transactions on Industrial Electronics.

[38]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.