Retrieval of Injection Molding Industrial Knowledge Graph Based on Transformer and BERT

Knowledge graphs play an important role in the field of knowledge management by providing a simple and clear way of expressing complex data relationships. Injection molding is a highly knowledge-intensive technology, and in our previous research, we have used knowledge graphs to manage and express relevant knowledge, gradually establishing an injection molding industrial knowledge graph. However, the current way of retrieving knowledge graphs is still mainly through programming, which results in many difficulties for users without programming backgrounds when it comes to searching a graph. This study will utilize the previously established injection molding industrial knowledge graph and employ a BERT (Bidirectional Encoder Representations from Transformers) fine-tuning model to analyze the semantics of user questions. A knowledge graph will be retrieved through a search engine built on the Transformer Encoder, which can reason based on the structure of the graph to find relevant knowledge that satisfies a user’s questions. The experimental results show that both the BERT fine-tuned model and the search engine achieve an excellent performance. This approach can help engineers who do not have a knowledge graph background to retrieve information from the graph by inputting natural language queries, thereby improving the usability of the graph.

[1]  S. Chen,et al.  Development and Application of Knowledge Graphs for the Injection Molding Process , 2023, Machines.

[2]  W. Jong,et al.  Knowledge Management for Injection Molding Defects by a Knowledge Graph , 2022, Applied Sciences.

[3]  Yibin Chen,et al.  A hybrid CNN-Transformer model for ozone concentration prediction , 2022, Air Quality, Atmosphere & Health.

[4]  Jie Chen,et al.  A transformer-based deep neural network for arrhythmia detection using continuous ECG signals , 2022, Comput. Biol. Medicine.

[5]  M. Jia,et al.  A Convolutional Transformer Architecture for Remaining Useful Life Estimation , 2021, 2021 Global Reliability and Prognostics and Health Management (PHM-Nanjing).

[6]  Jinrong Li,et al.  Systematic Knowledge Management of Construction Safety Standards Based on Knowledge Graphs: A Case Study in China , 2021, International journal of environmental research and public health.

[7]  B. E. Asri,et al.  BERT Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis , 2021, Computer Methods and Programs in Biomedicine Update.

[8]  Kun'ao Zhang,et al.  A greenhouse temperature prediction framework based on transformer neural network , 2021, 2021 International Conference on Computer Information Science and Artificial Intelligence (CISAI).

[9]  Yu-xin Shi,et al.  Constructing Crop Portraits Based on Graph Databases Is Essential to Agricultural Data Mining , 2021, Inf..

[10]  Xiaoyu Chen,et al.  Entity recognition model of power safety regulations knowledge graph based on BERT-BiLSTM-CRF , 2021, 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA).

[11]  D. Grechishnikova Transformer neural network for protein-specific de novo drug generation as a machine translation problem , 2021, Scientific Reports.

[12]  Yu Chen,et al.  Personalized Food Recommendation as Constrained Question Answering over a Large-scale Food Knowledge Graph , 2021, WSDM.

[13]  Yang Tang,et al.  A knowledge graph method for hazardous chemical management: Ontology design and entity identification , 2020, Neurocomputing.

[14]  U. Acikalin,et al.  Turkish Sentiment Analysis Using BERT , 2020, 2020 28th Signal Processing and Communications Applications Conference (SIU).

[15]  Xiangning Lin,et al.  Construction and Application of Knowledge Graph for Power System Dispatching , 2020, 2020 7th International Forum on Electrical Engineering and Automation (IFEEA).

[16]  Ye Wang,et al.  HKGB: An Inclusive, Extensible, Intelligent, Semi-auto-constructed Knowledge Graph Framework for Healthcare with Clinicians' Expertise Incorporated , 2020, Inf. Process. Manag..

[17]  Maria-Esther Vidal,et al.  A Knowledge Graph for Industry 4.0 , 2020, ESWC.

[18]  知秀 柴田 5分で分かる!? 有名論文ナナメ読み:Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding , 2020 .

[19]  Xue Ben,et al.  Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case , 2020, ArXiv.

[20]  Noel Crespi,et al.  A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media , 2019, COMPLEX NETWORKS.

[21]  Junrui Zhang,et al.  Research on knowledge graph for quantification of relationship between enterprises and recognition of potential risks , 2019, 2019 4th IEEE International Conference on Cybernetics (Cybconf).

[22]  Mohammad Reza Khosravani,et al.  Application of case-based reasoning in a fault detection system on production of drippers , 2019, Appl. Soft Comput..

[23]  Yang Chi,et al.  Knowledge Management in Healthcare Sustainability: A Smart Healthy Diet Assistant in Traditional Chinese Medicine Culture , 2018, Sustainability.

[24]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[25]  Ling Zhu,et al.  Knowledge graph for TCM health preservation: Design, construction, and applications , 2017, Artif. Intell. Medicine.

[26]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[27]  Tong Yu,et al.  Expert knowledge maps for knowledge management: a case study in Traditional Chinese Medicine research. , 2013, Journal of traditional Chinese medicine = Chung i tsa chih ying wen pan.

[28]  Wen-Ren Jong,et al.  An integrated application for historical knowledge management with mould design navigating process , 2013 .

[29]  João Carlos Espíndola Ferreira,et al.  A distributed system for rapid determination of nonconformance causes and solutions for the thermoplastic injection molding process: A Case-Based Reasoning Agents approach , 2011, 2011 IEEE International Conference on Automation Science and Engineering.

[30]  Wen-Ren Jong,et al.  Web-based navigating system for conceptual mould design with knowledge management , 2011 .

[31]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[32]  Peter Haddawy,et al.  Automatic Cause-Effect Relation Extraction from Dental Textbooks Using BERT , 2021, ICADL.

[33]  Haklae Kim,et al.  A knowledge graph of medical institutions in Korea , 2021 .

[34]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[35]  Jorge Bernardino,et al.  Graph Databases: Neo4j Analysis , 2017, ICEIS.