Hierarchical Invariant Learning for Domain Generalization Recommendation

Most cross-domain recommenders require samples on target domains or source-target overlaps to carry out domain adaptation. However, in many real-world situations, target domains are lack of such knowledge. Few works discuss this problem, whose essence is domain generalization recommendation. In this paper, we figure out domain generalization recommendation with a clear symbolized definition and propose corresponding models. Moreover, we illustrate its strong connection with zero-shot recommendation, pretrained recommendation and cold-start recommendation, distinguishing it from content-based recommendation. By analyzing its properties, we propose HIRL^+ and a series of heuristic methods to solve this problem. We propose hierarchical invariant learning to expel the specific patterns in both domain-level and environment-level, and find the common patterns in generalization space. To make the division of environments flexible, fine-grained and balanced, we put forward a learnable environment assignment method. To improve the robustness against distribution shifts inside domain generalization, we present an adversarial environment refinement method. In addition, we conduct experiments on real-word datasets to verify the effectiveness of our models, and carry out further studies on the domain distance and domain diversity. To benefit the research community and promote this direction, we discuss the future of this field.

[1]  Wayne Xin Zhao,et al.  Towards Universal Sequence Representation Learning for Recommender Systems , 2022, KDD.

[2]  E. Bareinboim,et al.  Causal Transportability for Visual Recognition , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Min Lin,et al.  Causal Representation Learning for Out-of-Distribution Recommendation , 2022, WWW.

[4]  Shirui Pan,et al.  MAMDR: A Model Agnostic Learning Method for Multi-Domain Recommendation , 2022, ArXiv.

[5]  Xiangnan He,et al.  KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems , 2022, CIKM.

[6]  Yue He,et al.  CausPref: Causal Preference Learning for Out-of-Distribution Recommendation , 2022, WWW.

[7]  E. Xing,et al.  Towards Principled Disentanglement for Domain Generalization , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Fuzhen Zhuang,et al.  Personalized Transfer of User Preferences for Cross-domain Recommendation , 2021, WSDM.

[9]  Jiadi Yu,et al.  A Survey on Cross-domain Recommendation: Taxonomies, Methods, and Future Directions , 2021, ACM Trans. Inf. Syst..

[10]  Jiwen Lu,et al.  Human Trajectory Prediction via Counterfactual Analysis , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Zachary Chase Lipton,et al.  Explaining The Efficacy of Counterfactually-Augmented Data , 2020, ICLR.

[12]  Amit Sharma,et al.  Domain Generalization using Causal Matching , 2020, ICML.

[13]  Hongbo Deng,et al.  CATN: Cross-Domain Recommendation for Cold-Start Users via Aspect Transfer Network , 2020, SIGIR.

[14]  Yifan Zhang,et al.  Correcting for Selection Bias in Learning-to-rank Systems , 2020, WWW.

[15]  Tatsunori B. Hashimoto,et al.  Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization , 2019, ArXiv.

[16]  Dongha Lee,et al.  Semi-Supervised Learning for Cross-Domain Recommendation to Cold-Start Users , 2019, CIKM.

[17]  Chang-Dong Wang,et al.  LSCD: Low-rank and sparse cross-domain recommendation , 2019, Neurocomputing.

[18]  Pan Li,et al.  DDTCDR: Deep Dual Transfer Cross Domain Recommendation , 2019, WSDM.

[19]  Markus Schedl,et al.  Deep Learning in Music Recommendation Systems , 2019, Front. Appl. Math. Stat..

[20]  Yang Xu,et al.  Deeply Fusing Reviews and Contents for Cold Start Users in Cross-Domain Recommendation Systems , 2019, AAAI.

[21]  David Lopez-Paz,et al.  Invariant Risk Minimization , 2019, ArXiv.

[22]  Xiaowei Wang,et al.  Sequential Scenario-Specific Meta Learner for Online Recommendation , 2019, KDD.

[23]  Lina Yao,et al.  DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patterns , 2019, IJCAI.

[24]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[25]  Jenq-Neng Hwang,et al.  Solving the Sparsity Problem in Recommendations via Cross-Domain Item Embedding Based on Co-Clustering , 2019, WSDM.

[26]  John C. Duchi,et al.  Learning Models with Uniform Performance via Distributionally Robust Optimization , 2018, ArXiv.

[27]  Chang Zhou,et al.  Deep Interest Evolution Network for Click-Through Rate Prediction , 2018, AAAI.

[28]  Alex ChiChung Kot,et al.  Domain Generalization with Adversarial Feature Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Taiji Suzuki,et al.  Cross-domain Recommendation via Deep Domain Adaptation , 2018, ECIR.

[30]  Tat-Seng Chua,et al.  Cross-Domain Recommendation via Clustering on Multi-Layer Graphs , 2017, SIGIR.

[31]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[32]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[33]  Chih-Jen Lin,et al.  Field-aware Factorization Machines for CTR Prediction , 2016, RecSys.

[34]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[35]  Jun Wang,et al.  Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction , 2016, ECIR.

[36]  Bernhard Schölkopf,et al.  Invariant Models for Causal Transfer Learning , 2015, J. Mach. Learn. Res..

[37]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[38]  Flemming Topsøe,et al.  Jensen-Shannon divergence and Hilbert space embedding , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[39]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[40]  Jakub M. Tomczak,et al.  Selecting Data Augmentation for Simulating Interventions , 2021, ICML.

[41]  James M. Joyce.  Kullback-Leibler Divergence , 2011, International Encyclopedia of Statistical Science.

[42]  R. Lewis.  An Introduction to Classification and Regression Tree (CART) Analysis , 2000 .