Stratified and time-aware sampling based adaptive ensemble learning for streaming recommendations

Recommender systems have played an increasingly important role in providing users with tailored suggestions based on their preferences. However, the conventional offline recommender systems cannot handle the ubiquitous data stream well. To address this issue, Streaming Recommender Systems (SRSs) have emerged in recent years, which incrementally train recommendation models on newly received data for effective real-time recommendations. Focusing on new data only benefits addressing concept drift, i.e., the changing user preferences towards items. However, it impedes capturing long-term user preferences. In addition, the commonly existing underload and overload problems should be well tackled for higher accuracy of streaming recommendations. To address these problems, we propose a Stratified and Time-aware Sampling based Adaptive Ensemble Learning framework, called STS-AEL, to improve the accuracy of streaming recommendations. In STS-AEL, we first devise stratified and time-aware sampling to extract representative data from both new data and historical data to address concept drift while capturing long-term user preferences. Also, incorporating the historical data benefits utilizing the idle resources in the underload scenario more effectively. After that, we propose adaptive ensemble learning to efficiently process the overloaded data in parallel with multiple individual recommendation models, and then effectively fuse the results of these models with a sequential adaptive mechanism. Extensive experiments conducted on three real-world datasets demonstrate that STS-AEL, in all the cases, significantly outperforms the state-of-the-art SRSs.

[1]  Philip S. Yu,et al.  Mining concept-drifting data streams using ensemble classifiers , 2003, KDD '03.

[2]  Andreas Lommatzsch,et al.  Real-Time News Recommendation Using Context-Aware Ensembles , 2014, ECIR.

[3]  Tian Tian,et al.  Collaborative filtering recommendation algorithm integrating time windows and rating predictions , 2019, Applied Intelligence.

[4]  Yan Wang,et al.  DTCDR: A Framework for Dual-Target Cross-Domain Recommendation , 2019, CIKM.

[5]  Argimiro Arratia,et al.  GeoSRS: A hybrid social recommender system for geolocated data , 2016, Inf. Syst..

[6]  João Gama,et al.  Fast Incremental Matrix Factorization for Recommendation with Positive-Only Feedback , 2014, UMAP.

[7]  Yanchun Zhang,et al.  Multi-Window Based Ensemble Learning for Classification of Imbalanced Streaming Data , 2015, WISE.

[8]  Carlo Zaniolo,et al.  Fast and Light Boosting for Adaptive Mining of Data Streams , 2004, PAKDD.

[9]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.

[10]  DeLiang Wang,et al.  A Deep Ensemble Learning Method for Monaural Speech Separation , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[11]  Dimitris Plexousakis,et al.  Incremental Collaborative Filtering for Highly-Scalable Recommendation Algorithms , 2005, ISMIS.

[12]  Gary Lee Welcome to Cloud Networking , 2014 .

[13]  Craig MacDonald,et al.  A Deep Recurrent Collaborative Filtering Framework for Venue Recommendation , 2017, CIKM.

[14]  Sivaramakrishnan Natarajan,et al.  Enhancing recommendation stability of collaborative filtering recommender system through bio-inspired clustering ensemble method , 2018, Neural Computing and Applications.

[15]  Zi Huang,et al.  Streaming Ranking Based Recommender Systems , 2018, SIGIR.

[16]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[17]  Ruben Mayer,et al.  A Comprehensive Survey on Parallelization and Elasticity in Stream Processing , 2019, ACM Comput. Surv..

[18]  Mehmed M. Kantardzic,et al.  On the reliable detection of concept drift from streaming unlabeled data , 2017, Expert Syst. Appl..

[19]  Yi Li,et al.  A hybrid recommendation algorithm adapted in e-learning environments , 2012, World Wide Web.

[20]  François Fleuret,et al.  Reservoir Boosting : Between Online and Offline Ensemble Learning , 2013, NIPS.

[21]  Alejandro Bellogín,et al.  Content-based recommendation in social tagging systems , 2010, RecSys '10.

[22]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[23]  Mehregan Mahdavi,et al.  A social recommender system using item asymmetric correlation , 2018, Applied Intelligence.

[24]  Lina Yao,et al.  Deep Learning Based Recommender System , 2017, ACM Comput. Surv..

[25]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[26]  Pasquale Lops,et al.  Content-based Recommender Systems: State of the Art and Trends , 2011, Recommender Systems Handbook.

[27]  Zi Huang,et al.  Neural Memory Streaming Recommender Networks with Adversarial Training , 2018, KDD.

[28]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Unil Yun,et al.  Sliding window based weighted erasable stream pattern mining for stream data applications , 2016, Future Gener. Comput. Syst..

[31]  WangDeLiang,et al.  A deep ensemble learning method for monaural speech separation , 2016 .

[32]  Ahmad A. Kardan,et al.  A novel approach to hybrid recommendation systems based on association rules mining for content recommendation in asynchronous discussion groups , 2013, Inf. Sci..

[33]  Luis Martínez,et al.  Fuzzy Tools in Recommender Systems: A Survey , 2017, Int. J. Comput. Intell. Syst..

[34]  Wei Wang,et al.  Recommender system application developments: A survey , 2015, Decis. Support Syst..

[35]  Naoto Yokoya,et al.  Random Forest Ensembles and Extended Multiextinction Profiles for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Stuart J. Russell,et al.  Online bagging and boosting , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[37]  Lars Schmidt-Thieme,et al.  Real-time top-n recommendation in social streams , 2012, RecSys.

[38]  Hongzhi Yin,et al.  Streaming Session-based Recommendation , 2019, KDD.

[39]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[40]  Feng Zhu,et al.  A Deep Framework for Cross-Domain and Cross-System Recommendations , 2018, IJCAI.

[41]  Rui Araújo,et al.  An on-line weighted ensemble of regressor models to handle concept drifts , 2015, Eng. Appl. Artif. Intell..

[42]  Yongluan Zhou,et al.  Dynamic Resource Management In a Massively Parallel Stream Processing Engine , 2015, CIKM.

[43]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[44]  Lin Li,et al.  Streaming Recommendation Algorithm with User Interest Drift Analysis , 2019, APWeb/WAIM.

[45]  Geoff Holmes,et al.  Leveraging Bagging for Evolving Data Streams , 2010, ECML/PKDD.

[46]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[47]  Lawrence Carin,et al.  Active learning for online bayesian matrix factorization , 2012, KDD.

[48]  Elisabetta Fersini,et al.  Sentiment analysis: Bayesian Ensemble Learning , 2014, Decis. Support Syst..

[49]  Mohsen Rahmani,et al.  A recommender system for tourism industry using cluster ensemble and prediction machine learning techniques , 2017, Comput. Ind. Eng..

[50]  Min Wu,et al.  Online Collaborative Filtering with Implicit Feedback , 2019, DASFAA.

[51]  Jun Li,et al.  ${{\rm E}^{2}}{\rm LMs}$ : Ensemble Extreme Learning Machines for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[52]  Peter R. Pietzuch,et al.  THEMIS: Fairness in Federated Stream Processing under Overload , 2016, SIGMOD Conference.

[53]  Lars Schmidt-Thieme,et al.  Online-updating regularized kernel matrix factorization models for large-scale recommender systems , 2008, RecSys '08.

[54]  Jean Paul Barddal,et al.  A Survey on Ensemble Learning for Data Stream Classification , 2017, ACM Comput. Surv..

[55]  A KardanAhmad,et al.  A novel approach to hybrid recommendation systems based on association rules mining for content recommendation in asynchronous discussion groups , 2013 .

[56]  Quan Z. Sheng,et al.  Modeling Multi-Purpose Sessions for Next-Item Recommendations via Mixture-Channel Purpose Routing Networks , 2019, IJCAI.

[57]  Nicolas Kourtellis,et al.  Dynamic Matrix Factorization with Priors on Unknown Values , 2015, KDD.

[58]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[59]  Tao Chen,et al.  TriRank: Review-aware Explainable Recommendation by Modeling Aspects , 2015, CIKM.

[60]  Chih-Fong Tsai,et al.  Cluster ensembles in collaborative filtering recommendation , 2012, Appl. Soft Comput..

[61]  Bin Zhou,et al.  Multi-window based ensemble learning for classification of imbalanced streaming data , 2015, World Wide Web.

[62]  João Gama,et al.  Ensemble learning for data stream analysis: A survey , 2017, Inf. Fusion.

[63]  Li Deng,et al.  Ensemble deep learning for speech recognition , 2014, INTERSPEECH.

[64]  Hong Shen,et al.  Neural variational matrix factorization for collaborative filtering in recommendation systems , 2019, Applied Intelligence.

[65]  Axel-Cyrille Ngonga Ngomo,et al.  Ensemble Learning for Named Entity Recognition , 2014, SEMWEB.

[66]  Quan Z. Sheng,et al.  Sequential Recommender Systems: Challenges, Progress and Prospects , 2019, IJCAI.

[67]  Longbing Cao,et al.  Perceiving the Next Choice with Comprehensive Transaction Embeddings for Online Recommendation , 2017, ECML/PKDD.

[68]  WangWei,et al.  Recommender system application developments , 2015 .

[69]  Walter Karlen,et al.  Granger-Causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks , 2018, AAAI.

[70]  DTCDR , 2019, Proceedings of the 28th ACM International Conference on Information and Knowledge Management.

[71]  Gavin Brown,et al.  Ensemble Learning , 2010, Encyclopedia of Machine Learning and Data Mining.

[72]  Guangjie Han,et al.  Resource-utilization-aware energy efficient server consolidation algorithm for green computing in IIOT , 2018, J. Netw. Comput. Appl..

[73]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[74]  Yi Chang,et al.  Streaming Recommender Systems , 2016, WWW.