A microservice-based framework for exploring data selection in cross-building knowledge transfer

Supervised deep learning has achieved remarkable success in various applications. Successful machine learning application however depends on the availability of sufficiently large amount of data. In the absence of data from the target domain, representative data collection from multiple sources is often needed. However, a model trained on existing multi-source data might generalize poorly on the unseen target domain. This problem is referred to as domain shift. In this paper, we explore the suitability of multi-source training data selection to tackle the domain shift challenge in the context of domain generalization. We also propose a microservice-oriented methodology for supporting this solution. We perform our experimental study on the use case of building energy consumption prediction. Experimental results suggest that minimal building description is capable of improving cross-building generalization performances when used to select energy consumption data.

[1]  Rama Chellappa,et al.  Wasserstein Distance Based Domain Adaptation for Object Detection , 2019, ArXiv.

[2]  Masashi Sugiyama,et al.  Mixture Regression for Covariate Shift , 2006, NIPS.

[3]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[4]  Pascal Fua,et al.  Beyond Sharing Weights for Deep Domain Adaptation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Yun Fu,et al.  Deep Domain Generalization With Structured Low-Rank Constraint. , 2018, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[6]  Meiqin Liu,et al.  Short-Term Load Forecasting with Multi-Source Data Using Gated Recurrent Unit Neural Networks , 2018 .

[7]  Yun Fu,et al.  Deep Domain Generalization With Structured Low-Rank Constraint , 2018, IEEE Transactions on Image Processing.

[8]  David J. Hill,et al.  Short-Term Residential Load Forecasting Based on Resident Behaviour Learning , 2018, IEEE Transactions on Power Systems.

[9]  Yongxin Yang,et al.  Multivariate Regression on the Grassmannian for Predicting Novel Domains , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Ivor W. Tsang,et al.  Domain adaptation from multiple sources via auxiliary classifiers , 2009, ICML '09.

[11]  Sozo Inoue,et al.  A personalized load forecasting enhanced by activity information , 2015, 2015 IEEE First International Smart Cities Conference (ISC2).

[12]  Dong Xu,et al.  Exploiting Low-Rank Structure from Latent Domains for Domain Generalization , 2014, ECCV.

[13]  V. Stanković,et al.  An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study , 2017, Scientific Data.

[14]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[15]  Hans-Peter Kriegel,et al.  Integrating structured biological data by Kernel Maximum Mean Discrepancy , 2006, ISMB.

[16]  Bernhard Schölkopf,et al.  Domain Generalization via Invariant Feature Representation , 2013, ICML.

[17]  Henk Visscher,et al.  Effectiveness of energy renovations: a reassessment based on actual consumption savings , 2018, Energy Efficiency.

[18]  Mengjie Zhang,et al.  Domain Generalization for Object Recognition with Multi-task Autoencoders , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Motoaki Kawanabe,et al.  Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , 2007, NIPS.

[20]  Yongxin Yang,et al.  Deeper, Broader and Artier Domain Generalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Jason Runge,et al.  Forecasting Energy Use in Buildings Using Artificial Neural Networks: A Review , 2019, Energies.

[22]  Yuan Zhang,et al.  Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network , 2019, IEEE Transactions on Smart Grid.

[23]  Gilles Blanchard,et al.  Generalizing from Several Related Classification Tasks to a New Unlabeled Sample , 2011, NIPS.

[24]  Tao Qin,et al.  Learning What Data to Learn , 2017, ArXiv.

[25]  Barbara Caputo,et al.  Best Sources Forward: Domain Generalization through Source-Specific Nets , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[26]  Yongxin Yang,et al.  Learning to Generalize: Meta-Learning for Domain Generalization , 2017, AAAI.

[27]  Daniel L. Marino,et al.  Building energy load forecasting using Deep Neural Networks , 2016, IECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society.

[28]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[29]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[30]  Jianqiang Yi,et al.  Building Energy Consumption Prediction: An Extreme Deep Learning Approach , 2017 .

[31]  Zachary Chase Lipton.  A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[32]  Daniel L. Marino,et al.  Deep neural networks for energy load forecasting , 2017, 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE).

[33]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[34]  Wouter M. Kouw,et al.  A Review of Domain Adaptation without Target Labels , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jian Shen,et al.  Wasserstein Distance Guided Representation Learning for Domain Adaptation , 2017, AAAI.

[36]  Yongxin Yang,et al.  A Unified Perspective on Multi-Domain and Multi-Task Learning , 2014, ICLR.

[37]  Ivan Glesk,et al.  Machine learning for estimation of building energy consumption and performance: a review , 2018, Visualization in Engineering.

[38]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[39]  Sung-Bae Cho,et al.  Predicting residential energy consumption using CNN-LSTM neural networks , 2019, Energy.

[40]  George Trigeorgis,et al.  Domain Separation Networks , 2016, NIPS.

[41]  Alex ChiChung Kot,et al.  Domain Generalization with Adversarial Feature Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Alexei A. Efros,et al.  Undoing the Damage of Dataset Bias , 2012, ECCV.

[43]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Thomas G. Dietterich,et al.  To transfer or not to transfer , 2005, NIPS 2005.

[45]  Dong Xu,et al.  Exploiting web images for event recognition in consumer videos: A multiple source domain adaptation approach , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Jian Ma,et al.  A Deep Neural Network Model for Short-Term Load Forecast Based on Long Short-Term Memory Network and Convolutional Neural Network , 2018, Energies.

[47]  Swami Sankaranarayanan,et al.  MetaReg: Towards Domain Generalization using Meta-Regularization , 2018, NeurIPS.

[48]  Shahrokh Valaee,et al.  Recent Advances in Recurrent Neural Networks , 2017, ArXiv.

[49]  John G. Breslin,et al.  Data Selection Strategies for Multi-Domain Sentiment Analysis , 2017, ArXiv.