Single-shot Feature Selection for Multi-task Recommendations

Multi-task Recommender Systems (MTRSs) has become increasingly prevalent in a variety of real-world applications due to their exceptional training efficiency and recommendation quality. However, conventional MTRSs often input all relevant feature fields without distinguishing their contributions to different tasks, which can lead to confusion and a decline in performance. Existing feature selection methods may neglect task relations or require significant computation during model training in multi-task setting. To this end, this paper proposes a novel Single-shot Feature Selection framework for MTRSs, referred to as MultiSFS, which is capable of selecting feature fields for each task while considering task relations in a single-shot manner. Specifically, MultiSFS first efficiently obtains task-specific feature importance through a single forward-backward pass. Then, a data-task bipartite graph is constructed to learn field-level task relations. Subsequently, MultiSFS merges the feature importance according to task relations and selects feature fields for different tasks. To demonstrate the effectiveness and properties of MultiSFS, we integrate it with representative MTRS models and evaluate on three real-world datasets. The implementation code is available online to ease reproducibility.

[1]  J. Leskovec,et al.  Relational Multi-Task Learning: Modeling Relations between Data and Tasks , 2023, ICLR.

[2]  Yejing Wang,et al.  AutoDenoise: Automatic Data Instance Denoising for Recommendations , 2023, WWW.

[3]  Xiangyu Zhao,et al.  AutoMLP: Automated MLP for Sequential Recommendations , 2023, WWW.

[4]  Charu C. Aggarwal,et al.  Automated Feature Selection: A Reinforcement Learning Perspective , 2023, IEEE Transactions on Knowledge and Data Engineering.

[5]  Weinan Zhang,et al.  AutoGen: An Automated Dynamic Model Generation Framework for Recommender System , 2023, WSDM.

[6]  Kun Gai,et al.  Multi-Task Recommendations with Reinforcement Learning , 2023, WWW.

[7]  K. Tan,et al.  An Evolutionary Multitasking Algorithm With Multiple Filtering for High-Dimensional Feature Selection , 2022, IEEE Transactions on Evolutionary Computation.

[8]  Ruiming Tang,et al.  Intent-aware Multi-source Contrastive Alignment for Tag-enhanced Recommendation , 2022, 2023 IEEE 39th International Conference on Data Engineering (ICDE).

[9]  Ruiming Tang,et al.  AutoAssign: Automatic Shared Embedding Assignment in Streaming Recommendation , 2022, 2022 IEEE International Conference on Data Mining (ICDM).

[10]  Linjian Mo,et al.  CFS-MTL: A Causal Feature Selection Mechanism for Multi-task Learning via Pseudo-intervention , 2022, CIKM.

[11]  Fajie Yuan,et al.  Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems , 2022, NeurIPS.

[12]  Yuan Zhang,et al.  KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos , 2022, CIKM.

[13]  X. Wu,et al.  AdaFS: Adaptive Feature Selection in Deep Recommender System , 2022, KDD.

[14]  Shen Xin,et al.  Prototype Feature Extraction for Multi-task Learning , 2022, WWW.

[15]  Ting Bai,et al.  A Contrastive Sharing Model for Multi-Task Recommendation , 2022, The Web Conference.

[16]  Tong Xu,et al.  AutoField: Automating Feature Selection in Deep Recommender Systems , 2022, WWW.

[17]  Hongzhi Yin,et al.  Single-shot Embedding Dimension Search in Recommender System , 2022, SIGIR.

[18]  Jia-Bin Huang,et al.  ESCM2: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation , 2022, SIGIR.

[19]  Xiangyu Zhao.  Adaptive and automated deep recommender systems , 2022, SIGWEB Newsl..

[20]  Zhaoxin Huan,et al.  MSSM: A Multiple-level Sparse Sharing Model for Efficient Multi-Task Learning , 2021, SIGIR.

[21]  Jiliang Tang,et al.  AutoLoss: Automated Loss Function Search in Recommendations , 2021, KDD.

[22]  Xiaorui Liu,et al.  Automated Self-Supervised Learning for Graphs , 2021, ICLR.

[23]  Fuzhen Zhuang,et al.  Learning to Expand Audience via Meta Hybrid Experts and Critics for Recommendation and Advertising , 2021, KDD.

[24]  Fuzhen Zhuang,et al.  Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising , 2021, KDD.

[25]  Jiliang Tang,et al.  AutoDim: Field-aware Embedding Dimension Searchin Recommender Systems , 2021, WWW.

[26]  Nicholas D. Lane,et al.  Zero-Cost Proxies for Lightweight NAS , 2021, ICLR.

[27]  Qing Da,et al.  Delayed Feedback Modeling for the Entire Space Conversion Rate Prediction , 2020, ArXiv.

[28]  Junning Liu,et al.  Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations , 2020, RecSys.

[29]  Minghao Yin,et al.  Simplifying Reinforced Feature Selection via Restructured Choice Strategy of Single Agent , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[30]  Yong Ge,et al.  AutoFS: Automated Feature Selection via Diversity-aware Interactive Reinforcement Learning , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[31]  Jiliang Tang,et al.  Automated Embedding Size Search in Deep Recommender Systems , 2020, SIGIR.

[32]  Xiangnan He,et al.  Disentangled Graph Collaborative Filtering , 2020, SIGIR.

[33]  Jiliang Tang,et al.  AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations , 2020, ArXiv.

[34]  Pengfei Liu,et al.  Learning Sparse Sharing Architectures for Multiple Tasks , 2019, AAAI.

[35]  Jing Zhang,et al.  Entire Space Multi-Task Modeling via Post-Click Behavior Decomposition for Conversion Rate Prediction , 2019, SIGIR.

[36]  Jiliang Tang,et al.  DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems , 2019, AAAI.

[37]  Li-Jia Li,et al.  Feature Partitioning for Efficient Multi-Task Architectures , 2019, ArXiv.

[38]  Yanjie Fu,et al.  Automating Feature Subspace Exploration via Multi-Agent Reinforcement Learning , 2019, KDD.

[39]  Ed H. Chi,et al.  SNR: Sub-Network Routing for Flexible Parameter Sharing in Multi-Task Learning , 2019, AAAI.

[40]  Philip H. S. Torr,et al.  SNIP: Single-shot Network Pruning based on Connection Sensitivity , 2018, ICLR.

[41]  Zhe Zhao,et al.  Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts , 2018, KDD.

[42]  Liang Zhang,et al.  Deep reinforcement learning for page-wise recommendations , 2018, RecSys.

[43]  Xiao Ma,et al.  Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate , 2018, SIGIR.

[44]  Liang Zhang,et al.  Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning , 2018, KDD.

[45]  Liang Zhang,et al.  Deep Reinforcement Learning for List-wise Recommendations , 2017, ArXiv.

[46]  Ruoxi Wang,et al.  Deep & Cross Network for Ad Click Predictions , 2017, ADKDD@KDD.

[47]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[48]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[49]  Sebastian Ruder,et al.  Sluice networks: Learning what to share between loosely related tasks , 2017, arXiv.org.

[50]  Percy Liang,et al.  Understanding Black-box Predictions via Influence Functions , 2017, ICML.

[51]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[52]  Joachim Bingel,et al.  Identifying beneficial task relations for multi-task learning in deep neural networks , 2017, EACL.

[53]  Yu Zhang,et al.  Learning Sparse Task Relations in Multi-Task Learning , 2017, AAAI.

[54]  Weinan Zhang,et al.  Product-Based Neural Networks for User Response Prediction , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[55]  A. Gupta,et al.  Cross-Stitch Networks for Multi-task Learning , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Jun Wang,et al.  Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction , 2016, ECIR.

[57]  Shuicheng Yan,et al.  Visual Classification With Multitask Joint Sparse Representation , 2012, IEEE Transactions on Image Processing.

[58]  Qiang Yang,et al.  Multi-task learning for cross-platform siRNA efficacy prediction: an in-silico study , 2010, BMC Bioinformatics.

[59]  C. Furlanello,et al.  Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products , 2006 .

[60]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[61]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[62]  Huan Liu,et al.  Chi2: feature selection and discretization of numeric attributes , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[63]  Ho Kei Cheng,et al.  Task Aware Feature Extraction Framework for Sequential Dependence Multi-Task Learning , 2023 .

[64]  L. Breiman.  Arcing the edge , 1997 .

[65]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .