A Federated Learning-Based Industrial Health Prognostics for Heterogeneous Edge Devices Using Matched Feature Extraction

Data-driven industrial health prognostics require rich training data to develop accurate and reliable predictive models. However, stringent data privacy laws and the abundance of edge industrial data necessitate decentralized data utilization. Thus, the industrial health prognostics field is well suited to significantly benefit from federated learning (FL), a decentralized and privacy-preserving learning technique. However, FL-based health prognostics tasks have hardly been investigated due to the complexities of meaningfully aggregating model parameters trained from heterogeneous data to form a high performing federated model. Specifically, data heterogeneity among edge devices, stemming from dissimilar degradation mechanisms and unequal dataset sizes, poses a critical statistical challenge for developing accurate federated models. We propose a pioneering FL-based health prognostic model with a feature similarity-matched parameter aggregation algorithm to discriminatingly learn from heterogeneous edge data. The algorithm searches across the heterogeneous locally trained models and matches neurons with probabilistically similar feature extraction functions first, before selectively averaging them to form the federated model parameters. As the algorithm only averages similar neurons, as opposed to conventional naive averaging of coordinate-wise neurons, the distinct feature extractors of local models are carried over with less dilution to the resultant federated model. Using both cyclic degradation data of Li-ion batteries and non-cyclic data of turbofan engines, we demonstrate that the proposed method yields accuracy improvements as high as 44.5\% and 39.3\% for state-of-health estimation and remaining useful life estimation, respectively.

[1]  Min Wu,et al.  A Hybrid Ensemble Deep Learning Approach for Early Prediction of Battery Remaining Useful Life , 2023, IEEE/CAA Journal of Automatica Sinica.

[2]  Chau Yuen,et al.  Digital Twin for Real-time Li-Ion Battery State of Health Estimation With Partially Discharged Cycling Data , 2022, IEEE Transactions on Industrial Informatics.

[3]  Xiaoli Li,et al.  Slow-Varying Dynamics-Assisted Temporal Capsule Network for Machinery Remaining Useful Life Estimation , 2022, IEEE Transactions on Cybernetics.

[4]  Dong Shaojiang,et al.  Deep Transfer Learning Based on Bi-LSTM and Attention for Remaining Useful Life Prediction of Rolling Bearing , 2022, Reliability Engineering & System Safety.

[5]  A. Abusorrah,et al.  A survey of transfer learning for machinery diagnostics and prognostics , 2022, Artificial Intelligence Review.

[6]  B. Tang,et al.  Remaining useful life prediction of rolling bearing based on multi-head attention embedded Bi-LSTM network , 2022, Measurement.

[7]  Weihong Guo,et al.  Process Monitoring and Fault Prediction in Multivariate Time Series Using Bag-of-Words , 2022, IEEE Transactions on Automation Science and Engineering.

[8]  Zhenan Pang,et al.  An Age-Dependent and State-Dependent Adaptive Prognostic Approach for Hidden Nonlinear Degrading System , 2021, IEEE/CAA Journal of Automatica Sinica.

[9]  Ying Zheng,et al.  Remaining useful life prediction of lithium battery based on capacity regeneration point detection , 2021 .

[10]  Nathalie Baracaldo,et al.  Privacy-Preserving Machine Learning: Methods, Challenges and Directions , 2021, ArXiv.

[11]  S. Güttel,et al.  A comparison of LSTM and GRU networks for learning symbolic sequences , 2021, ArXiv.

[12]  Wei Zhang,et al.  Data privacy preserving federated transfer learning in machinery fault diagnostics using prior distributions , 2021, Structural Health Monitoring.

[13]  Aruna Seneviratne,et al.  Federated Learning for Internet of Things: A Comprehensive Survey , 2021, IEEE Communications Surveys & Tutorials.

[14]  Zhenghua Chen,et al.  Machine Remaining Useful Life Prediction via an Attention-Based Deep Learning Approach , 2021, IEEE Transactions on Industrial Electronics.

[15]  Amina Adadi A survey on data‐efficient algorithms in big data era , 2021, J. Big Data.

[16]  Yan Qin,et al.  Transfer Learning-Based State of Charge Estimation for Lithium-Ion Battery at Varying Ambient Temperatures , 2021, IEEE Transactions on Industrial Informatics.

[17]  Yan Qin,et al.  Time-Series Regeneration With Convolutional Recurrent Generative Adversarial Network for Remaining Useful Life Estimation , 2021, IEEE Transactions on Industrial Informatics.

[18]  M. Shamim Hossain,et al.  Deep Anomaly Detection for Time-Series Data in Industrial IoT: A Communication-Efficient On-Device Federated Learning Approach , 2020, IEEE Internet of Things Journal.

[19]  Vu Thanh Hien,et al.  Modeling Transmission Rate of COVID-19 in Regional Countries to Forecast Newly Infected Cases in a Nation by the Deep Learning Method , 2021, FDSE.

[20]  Yu Wang,et al.  Remaining Useful Life Prediction for Rolling Bearings Using EMD-RISI-LSTM , 2021, IEEE Transactions on Instrumentation and Measurement.

[21]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[22]  Li Li,et al.  A review of applications in federated learning , 2020, Comput. Ind. Eng..

[23]  Bernardete Ribeiro,et al.  Remaining Useful Life Estimation in Aircraft Components with Federated Learning , 2020 .

[24]  Qinghua Liu,et al.  Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization , 2020, NeurIPS.

[25]  Yi Tang,et al.  Privacy preservation for machine learning training and classification based on homomorphic encryption schemes , 2020, Inf. Sci..

[26]  Nicole Gruber,et al.  Are GRU Cells More Specific and LSTM Cells More Sensitive in Motive Classification of Text? , 2020, Frontiers in Artificial Intelligence.

[27]  Mihai V. Micea,et al.  Online state of health prediction method for lithium‐ion batteries, based on gated recurrent unit neural networks , 2020, International Journal of Energy Research.

[28]  Yasaman Khazaeni,et al.  Federated Learning with Matched Averaging , 2020, ICLR.

[29]  Anit Kumar Sahu,et al.  Federated Optimization in Heterogeneous Networks , 2018, MLSys.

[30]  Hee-Yeon Ryu,et al.  LSTM-Based Battery Remaining Useful Life Prediction With Multi-Channel Charging Profiles , 2020, IEEE Access.

[31]  Sathyan Munirathinam,et al.  Chapter Six - Industry 4.0: Industrial Internet of Things (IIOT) , 2020, Adv. Comput..

[32]  Chih-Hung Wu,et al.  Learning To Recognize Driving Patterns For Collectively Characterizing Electric Vehicle Driving Behaviors , 2019, International Journal of Automotive Technology.

[33]  Kai Wang,et al.  Remaining useful life prediction for supercapacitor based on long short-term memory neural network , 2019, Journal of Power Sources.

[34]  Hongseok Kim,et al.  Machine Learning-Based Lithium-Ion Battery Capacity Estimation Exploiting Multi-Channel Charging Profiles , 2019, IEEE Access.

[35]  Yasaman Khazaeni,et al.  Bayesian Nonparametric Federated Learning of Neural Networks , 2019, ICML.

[36]  Weiwen Peng,et al.  Estimation of Bearing Remaining Useful Life Based on Multiscale Convolutional Neural Network , 2019, IEEE Transactions on Industrial Electronics.

[37]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[38]  Ali Emadi,et al.  Long Short-Term Memory Networks for Accurate State-of-Charge Estimation of Li-ion Batteries , 2018, IEEE Transactions on Industrial Electronics.

[39]  Peng Wang,et al.  Long short-term memory for machine remaining life prediction , 2018, Journal of Manufacturing Systems.

[40]  Yaguo Lei,et al.  Machinery health prognostics: A systematic review from data acquisition to RUL prediction , 2018 .

[41]  H. Brendan McMahan,et al.  Learning Differentially Private Recurrent Language Models , 2017, ICLR.

[42]  Chetan Gupta,et al.  Long Short-Term Memory Network for Remaining Useful Life estimation , 2017, 2017 IEEE International Conference on Prognostics and Health Management (ICPHM).

[43]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[44]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[45]  Ruqiang Yan,et al.  Machine health monitoring with LSTM networks , 2016, 2016 10th International Conference on Sensing Technology (ICST).

[46]  Hermann Ney,et al.  LSTM, GRU, Highway and a Bit of Attention: An Empirical Overview for Language Modeling in Speech Recognition , 2016, INTERSPEECH.

[47]  Rakesh Nagi,et al.  GPU-accelerated Hungarian algorithms for the Linear Assignment Problem , 2016, Parallel Comput..

[48]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[49]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[50]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[51]  C. Dwork A firm foundation for private data analysis , 2011, Commun. ACM.

[52]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[53]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[54]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[55]  Ralph E. White,et al.  Capacity Fade Mechanisms and Side Reactions in Lithium‐Ion Batteries , 1998 .

[56]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[57]  R. Keith Mobley,et al.  An introduction to predictive maintenance , 1989 .

[58]  Ronald L. Rivest,et al.  ON DATA BANKS AND PRIVACY HOMOMORPHISMS , 1978 .