BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring

Non-intrusive load monitoring (NILM) based energy disaggregation is the decomposition of a system's energy into the consumption of its individual appliances. Previous work on deep learning NILM algorithms has shown great potential in the field of energy management and smart grids. In this paper, we propose BERT4NILM, an architecture based on bidirectional encoder representations from transformers (BERT) and an improved objective function designed specifically for NILM learning. We adapt the bidirectional transformer architecture to the field of energy disaggregation and follow the pattern of sequence-to-sequence learning. With the improved loss function and masked training, BERT4NILM outperforms state-of-the-art models across various metrics on the two publicly available datasets UK-DALE and REDD.

[1]  Hengxu Zhang,et al.  Regularized LSTM Based Deep Learning Model: First Step towards Real-Time Non-Intrusive Load Monitoring , 2018, 2018 IEEE International Conference on Smart Energy Grid Engineering (SEGE).

[2]  Razvan Pascanu,et al.  Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks , 2013, ECML/PKDD.

[3]  Jun Hu,et al.  Scale- and Context-Aware Convolutional Non-Intrusive Load Monitoring , 2019, IEEE Transactions on Power Systems.

[4]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[5]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[6]  Bin Yang,et al.  A novel DNN-HMM-based approach for extracting single loads from aggregate power signals , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Jack Kelly,et al.  The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes , 2014, Scientific Data.

[8]  Songsong Chen,et al.  A SVM Optimized by Particle Swarm Optimization Approach to Load Disaggregation in Non-Intrusive Load Monitoring in Smart Homes , 2019, 2019 IEEE 3rd Conference on Energy Internet and Energy System Integration (EI2).

[9]  Kevin Gimpel,et al.  Gaussian Error Linear Units (GELUs) , 2016 .

[10]  Jack Kelly,et al.  Neural NILM: Deep Neural Networks Applied to Energy Disaggregation , 2015, BuildSys@SenSys.

[11]  Veronica Piccialli,et al.  Non-Intrusive Load Monitoring with an Attention-based Deep Neural Network , 2019, ArXiv.

[12]  Ke Liu,et al.  Sequence-To-Subsequence Learning With Conditional Gan For Power Disaggregation , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  J. Zico Kolter,et al.  REDD : A Public Data Set for Energy Disaggregation Research , 2011 .

[14]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[15]  Shufang Li,et al.  Semisupervised Multilabel Deep Learning Based Nonintrusive Load Monitoring in Smart Grids , 2020, IEEE Transactions on Industrial Informatics.

[16]  Tommi S. Jaakkola,et al.  Approximate Inference in Additive Factorial HMMs with Application to Energy Disaggregation , 2012, AISTATS.

[17]  G. W. Hart,et al.  Nonintrusive appliance load monitoring , 1992, Proc. IEEE.

[18]  R. N. Elliott,et al.  American Council for an Energy-Efficient Economy , 2002 .

[19]  Vladimir Stankovic,et al.  Improving Event-Based Non-Intrusive Load Monitoring Using Graph Signal Processing , 2018, IEEE Access.

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Rongqing Zhang,et al.  Low-Rate Non-Intrusive Appliance Load Monitoring Based on Graph Signal Processing , 2019, 2019 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[22]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[23]  Charles A. Sutton,et al.  Sequence-to-point learning with neural networks for nonintrusive load monitoring , 2016, AAAI.

[24]  Corinna Fischer.  Feedback on household electricity consumption: a tool for saving energy? , 2008 .