A Dual Transformer Model for Intelligent Decision Support for Maintenance of Wind Turbines

Wind energy is one of the fastest-growing sustainable energy sources in the world but relies crucially on efficient and effective operations and maintenance to generate sufficient amounts of energy and reduce downtime of wind turbines and associated costs. Machine learning has been applied to fault prediction in wind turbines, but these predictions have not been supported with suggestions on how to avert and fix faults. We present a data-to-text generation system utilising transformers for generating corrective maintenance strategies for faults using SCADA data capturing the operational status of turbines. We achieve this in two stages: a first stage identifies faults based on SCADA input features and their relevance. A second stage performs content selection for the language generation task and creates maintenance strategies based on phrase-based natural language templates. Experiments show that our dual transformer model achieves an accuracy of up to 96.75% for alarm prediction and up to 75.35% for its choice of maintenance strategies during content-selection. A qualitative analysis shows that our generated maintenance strategies are promising. We make our human- authored maintenance templates publicly available, and include a brief video explaining our approach.

[1]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[2]  L. Devroye Non-Uniform Random Variate Generation , 1986 .

[3]  Zhifang Sui,et al.  Table-to-text Generation by Structure-aware Seq2seq Learning , 2017, AAAI.

[4]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[5]  Fernando Nogueira,et al.  Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning , 2016, J. Mach. Learn. Res..

[6]  Yingning Qiu,et al.  Monitoring wind turbine gearboxes , 2013 .

[7]  Christophe Gravier,et al.  Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triples , 2017, J. Web Semant..

[8]  David Infield,et al.  Online wind turbine fault detection through automated SCADA data analysis , 2009 .

[9]  Jesse Vig,et al.  A Multiscale Visualization of Attention in the Transformer Model , 2019, ACL.

[10]  Nina Dethlefs,et al.  Author response for "Deep learning with knowledge transfer for explainable anomaly prediction in wind turbines" , 2019 .

[11]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[12]  Nina Dethlefs,et al.  Evaluation of NLG in an end-to-end Spoken dialogue system- is it worth it? , 2016 .

[13]  Benjamin Graham Office,et al.  Random Sampling , 2019, Encyclopedic Dictionary of Archaeology.

[14]  Milica Gasic,et al.  Phrase-Based Statistical Language Generation Using Graphical Models and Active Learning , 2010, ACL.

[15]  Luowei Zhou,et al.  End-to-End Dense Video Captioning with Masked Transformer , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Erhardt Barth,et al.  A Hybrid Convolutional Variational Autoencoder for Text Generation , 2017, EMNLP.

[17]  Dahai Zhang,et al.  A data-driven approach for fault detection of offshore wind turbines using random forests , 2017, IECON 2017 - 43rd Annual Conference of the IEEE Industrial Electronics Society.

[18]  Alexander M. Rush,et al.  End-to-End Content and Plan Selection for Data-to-Text Generation , 2018, INLG.

[19]  Richard Socher,et al.  Pointer Sentinel Mixture Models , 2016, ICLR.

[20]  David McMillan,et al.  Availability, operation and maintenance costs of offshore wind turbines with different drive train configurations , 2017 .

[21]  Nina Dethlefs,et al.  Domain Transfer for Deep Natural Language Generation from Abstract Meaning Representations , 2017, IEEE Computational Intelligence Magazine.

[22]  Marta R. Costa-jussà,et al.  Neural Machine Translation with the Transformer and Multi-Source Romance Languages for the Biomedical WMT 2018 task , 2018, WMT.

[23]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.

[24]  Simon Hogg,et al.  Wind energy: UK experiences and offshore operational challenges , 2015 .

[25]  Nina Dethlefs,et al.  Natural Language Generation for Operations and Maintenance in Wind Turbines , 2019 .

[26]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[27]  Ondrej Dusek,et al.  Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Strings , 2016, ACL.

[28]  Zhiyu Chen,et al.  Few-shot NLG with Pre-trained Language Model , 2020, ACL.

[29]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[30]  Raymond J. Mooney,et al.  Training a Multilingual Sportscaster: Using Perceptual Context to Learn Language , 2014, J. Artif. Intell. Res..

[31]  Markus Freitag,et al.  Unsupervised Natural Language Generation with Denoising Autoencoders , 2018, EMNLP.

[32]  Rong Pan,et al.  Operation-guided Neural Networks for High Fidelity Data-To-Text Generation , 2018, EMNLP.

[33]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[34]  Ehud Reiter,et al.  Lessons from Deploying NLG Technology for Marine Weather Forecast Text Generation , 2004, ECAI.

[35]  David Vandyke,et al.  Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems , 2015, EMNLP.