Towards a Generic Framework for Mechanism-guided Deep Learning for Manufacturing Applications

Manufacturing data analytics tasks are traditionally undertaken with Mechanism Models (MMs), which are domain-specific mathematical equations modeling the underlying physical or chemical processes of the tasks. Recently, Deep Learning (DL) has been increasingly applied to manufacturing. MMs and DL have their individual pros and cons, motivating the development of Mechanism-guided Deep Learning Models (MDLMs) that combine the two. Existing MDLMs are often tailored to specific tasks or types of MMs, and can fail to effectively 1) utilize interconnections of multiple input examples, 2) adaptively self-correct prediction errors with error bounding, and 3) ensemble multiple MMs. In this work, we propose a generic, task-agnostic MDLM framework that can embed one or more MMs in deep networks, and address the 3 aforementioned issues. We present 2 diverse use cases where we experimentally demonstrate the effectiveness and efficiency of our models.

[1]  Yuntian Chen,et al.  AutoKE: An automatic knowledge embedding framework for scientific machine learning , 2022, IEEE Transactions on Artificial Intelligence.

[2]  R. Klein,et al.  Spline-PINN: Approaching PDEs without Data using Fast, Physics-Informed Hermite-Spline CNNs , 2021, AAAI.

[3]  D. Neuville,et al.  Structure and properties of alkali aluminosilicate glasses and melts: insights from deep learning , 2021, Geochimica et Cosmochimica Acta.

[4]  Anuj Karpatne,et al.  PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics , 2021, KDD.

[5]  Ning Liu,et al.  Beyond Outlier Detection: Outlier Interpretation by Attention-Guided Triplet Deviation Network , 2021, WWW.

[6]  Guangzhong Dong,et al.  Noise-Immune Model Identification and State-of-Charge Estimation for Lithium-Ion Battery Using Bilinear Parameterization , 2021, IEEE Transactions on Industrial Electronics.

[7]  Sotiris Kotsiantis,et al.  Explainable AI: A Review of Machine Learning Interpretability Methods , 2020, Entropy.

[8]  Hui Xiong,et al.  Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting , 2020, AAAI.

[9]  Noseong Park,et al.  DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation , 2020, AAAI.

[10]  Emmanuel Ramasso,et al.  Investigating computational geometry for failure prognostics , 2014, International Journal of Prognostics and Health Management.

[11]  Geoffrey I. Webb,et al.  Time series extrinsic regression , 2020, Data Mining and Knowledge Discovery.

[12]  Yaguo Lei,et al.  A Hybrid Prognostics Approach for Estimating Remaining Useful Life of Rolling Element Bearings , 2020, IEEE Transactions on Reliability.

[13]  Xiang Li,et al.  Diagnosing Rotating Machines With Weakly Supervised Data Using Deep Transfer Learning , 2020, IEEE Transactions on Industrial Informatics.

[14]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[15]  Chaomin Luo,et al.  Intelligent Demand Forecasting of Smelting Process Using Data-Driven and Mechanism Model , 2019, IEEE Transactions on Industrial Electronics.

[16]  Geoffrey I. Webb,et al.  ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels , 2019, Data Mining and Knowledge Discovery.

[17]  Geoffrey I. Webb,et al.  InceptionTime: Finding AlexNet for time series classification , 2019, Data Mining and Knowledge Discovery.

[18]  C. Bergmeir,et al.  Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions , 2019, International Journal of Forecasting.

[19]  James T. Kwok,et al.  Generalizing from a Few Examples , 2019, ACM Comput. Surv..

[20]  Linkan Bian,et al.  Deep Learning for Distortion Prediction in Laser-Based Additive Manufacturing using Big Data , 2019, Manufacturing Letters.

[21]  Yaguo Lei,et al.  A Wiener-Process-Model-Based Method for Remaining Useful Life Prediction Considering Unit-to-Unit Variability , 2019, IEEE Transactions on Industrial Electronics.

[22]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[23]  Germain Forestier,et al.  Deep learning for time series classification: a review , 2018, Data Mining and Knowledge Discovery.

[24]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[25]  Tao Shen,et al.  DiSAN: Directional Self-Attention Network for RNN/CNN-free Language Understanding , 2017, AAAI.

[26]  Yan Tong,et al.  Incremental Boosting Convolutional Neural Network for Facial Action Unit Recognition , 2017, NIPS.

[27]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[28]  Liang Guo,et al.  Remaining Useful Life Prediction Based on a General Expression of Stochastic Process Models , 2017, IEEE Transactions on Industrial Electronics.

[29]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Trevor Darrell,et al.  Fully convolutional networks for semantic segmentation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Vladimir M. Krasnopolsky,et al.  A Neural Network Nonlinear Multimodel Ensemble to Improve Precipitation Forecasts over Continental US , 2012 .

[33]  Scott McQuade,et al.  Global Climate Model Tracking Using Geospatial Neighborhoods , 2012, AAAI.

[34]  Claire Monteleoni,et al.  Tracking climate models , 2011, CIDU.

[35]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[36]  John C. Mauro,et al.  Viscosity of glass-forming liquids , 2009, Proceedings of the National Academy of Sciences.

[37]  L. Breiman.  Random Forests , 2001, Encyclopedia of Machine Learning and Data Mining.

[38]  Palmer,et al.  Nature of the glass transition. , 1988, Physical review. B, Condensed matter.

[39]  Andrey Milchev,et al.  Effect of disorder on diffusion and viscosity in condensed systems , 1988 .

[40]  Gary S. Grest,et al.  Liquid-glass transition, a free-volume approach , 1979 .

[41]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[42]  G. Adam,et al.  On the Temperature Dependence of Cooperative Relaxation Properties in Glass‐Forming Liquids , 1965 .

[43]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[44]  P. Zeiler,et al.  A Comparative Study on Methods for Fusing Data-Driven and Physics-Based Models for Hybrid Remaining Useful Life Prediction of Air Filters , 2023, IEEE Access.

[45]  Skipper Seabold,et al.  Statsmodels: Econometric and Statistical Modeling with Python , 2010, SciPy.

[46]  G. Grest,et al.  The nature of the glass transition , 1984 .

[47]  H. Vogel,et al.  Das Temperaturabhangigkeitsgesetz der Viskositat von Flussigkeiten , 1921 .