Domain Adaptation in Physical Systems via Graph Kernel

Physical systems are extending their monitoring capacities to edge areas with low-cost, low-power sensors and advanced data mining and machine learning techniques. However, new systems often have limited data for training the model, calling for effective knowledge transfer from other relevant grids. Specifically, Domain Adaptation (DA) seeks domain-invariant features to boost the model performance in the target domain. Nonetheless, existing DA techniques face significant challenges due to the unique characteristics of physical datasets: (1) complex spatial-temporal correlations, (2) diverse data sources including node/edge measurements and labels, and (3) large-scale data sizes. In this paper, we propose a novel cross-graph DA based on two core designs of graph kernels and graph coarsening. The former design handles spatial-temporal correlations and can incorporate networked measurements and labels conveniently. The spatial structures, temporal trends, measurement similarity, and label information together determine the similarity of two graphs, guiding the DA to find domain-invariant features. Mathematically, we construct a Graph kerNel-based distribution Adaptation (GNA) with a specifically-designed graph kernel. Then, we prove the proposed kernel is positive definite and universal, which strictly guarantees the feasibility of the used DA measure. However, the computation cost of the kernel is prohibitive for large systems. In response, we propose a novel coarsening process to obtain much smaller graphs for GNA. Finally, we report the superiority of GNA in diversified systems, including power systems, mass-damper systems, and human-activity sensing systems.

[1]  Yang Weng,et al.  A Transfer Learning Framework for Power System Event Identification , 2022, IEEE Transactions on Power Systems.

[2]  Yang Weng,et al.  Physical Equation Discovery Using Physics-Consistent Neural Network (PCNN) Under Incomplete Observability , 2021, KDD.

[3]  Youngeun Kim,et al.  Adaptive Graph Adversarial Networks for Partial Domain Adaptation , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Hanghang Tong,et al.  Heterogeneous Transfer Learning on Power Systems: A Merged Multi-Modal Gaussian Graphical Model , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[5]  Qi Chen,et al.  Beyond H-Divergence: Domain Adaptation Theory With Jensen-Shannon Divergence , 2020, ArXiv.

[6]  Guanrong Chen,et al.  NES-TL: Network Embedding Similarity-Based Transfer Learning , 2020, IEEE Transactions on Network Science and Engineering.

[7]  Kup-Sze Choi,et al.  Network Together: Node Classification via Cross-Network Deep Network Embedding , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Rama Chellappa,et al.  Wasserstein Distance Based Domain Adaptation for Object Detection , 2019, ArXiv.

[9]  Yiqiang Chen,et al.  Transfer Learning with Dynamic Adversarial Adaptation Network , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[10]  Xiao-Ming Wu,et al.  Graph Transfer Learning via Adversarial Domain Adaptation With Graph Convolution , 2019, IEEE Transactions on Knowledge and Data Engineering.

[11]  Yang Weng,et al.  An Unsupervised Learning Framework for Event Detection, Type Identification and Localization Using PMUs Without Any Historical Labels , 2019, 2019 IEEE Power & Energy Society General Meeting (PESGM).

[12]  Tianzhu Zhang,et al.  GCAN: Graph Convolutional Adversarial Network for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Yang Weng,et al.  A Hybrid Machine Learning Framework for Enhancing PMU-based Event Identification with Limited Labels , 2019, 2019 International Conference on Smart Grid Synchronized Measurements and Analytics (SGSMA).

[14]  Nils M. Kriege,et al.  A survey on graph kernels , 2019, Applied Network Science.

[15]  Lee M. Gunderson,et al.  A Unifying Framework for Spectrum-Preserving Graph Sparsification and Coarsening , 2019, NeurIPS.

[16]  Ming Shao,et al.  Graph Adaptive Knowledge Transfer for Unsupervised Domain Adaptation , 2018, ECCV.

[17]  Andreas Loukas,et al.  Graph Reduction with Spectral and Cut Guarantees , 2018, J. Mach. Learn. Res..

[18]  Hanghang Tong,et al.  FASTEN: Fast Sylvester Equation Solver for Graph Mining , 2018, KDD.

[19]  Firoj Alam,et al.  Domain Adaptation with Adversarial Training and Graph Embeddings , 2018, ACL.

[20]  Pierre Vandergheynst,et al.  Spectrally approximating large graphs with smaller graphs , 2018, ICML.

[21]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Sungroh Yoon,et al.  Transfer Learning for Deep Learning on Graph-Structured Data , 2016, AAAI.

[23]  Kate Saenko,et al.  Correlation Alignment for Unsupervised Domain Adaptation , 2016, Domain Adaptation in Computer Vision Applications.

[24]  Hanghang Tong,et al.  FINAL: Fast Attributed Network Alignment , 2016, KDD.

[25]  Fangzhao Wu,et al.  Sentiment Domain Adaptation with Multiple Sources , 2016, ACL.

[26]  Michael I. Jordan,et al.  Unsupervised Domain Adaptation with Residual Transfer Networks , 2016, NIPS.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[29]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[30]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[31]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Jie Yin,et al.  Transfer Learning across Networks for Collective Classification , 2013, 2013 IEEE 13th International Conference on Data Mining.

[33]  Mi Zhang,et al.  USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors , 2012, UbiComp.

[34]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[35]  Qiang Yang,et al.  Transfer Learning via Dimensionality Reduction , 2008, AAAI.

[36]  Karsten M. Borgwardt,et al.  Graph Kernels , 2008, J. Mach. Learn. Res..

[37]  Cheng Wu,et al.  A Graph Embedding Framework for Maximum Mean Discrepancy-Based Domain Adaptation Algorithms , 2020, IEEE Transactions on Image Processing.

[38]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[39]  MarchandMario,et al.  Domain-adversarial training of neural networks , 2016 .

[40]  K. Fukumizu,et al.  Learning via Hilbert Space Embedding of Distributions , 2007 .

[41]  Hans-Peter Kriegel,et al.  Protein function prediction via graph kernels , 2005, ISMB.