Due to the applications of user engagements in recommender systems, predicting user engagement has recently attracted considerable attention. In this task which is firstly proposed in ACM Recommender Systems Challenge 2014, the posts containing users’ opinions about items (e.g., the tweets containing the users’ ratings about movies in the IMDb website) are studied. In this paper, we focus on user engagement evaluation for cold-start web applications in the extreme case, when there is no training data available for the target web application. We propose an adaptive model based on transfer learning (TL) technique to train on the data from a web application and test on another one. We study the problem of detecting tweets with positive engagement, which is a highly imbalanced classification problem. Therefore, we modify the loss function of the employed transfer learning method to cope with imbalanced data. We evaluate our method using a dataset including the tweets of four popular and diverse data sources, i.e., IMDb, YouTube, Goodreads, and Pandora. The experimental results show that in some cases transfer learning can transfer knowledge among domains to improve the user engagement evaluation performance. We further analyze the results to figure out when transfer learning can help to improve the performance.
[1]
Azadeh Shakery,et al.
Regression and Learning to Rank Aggregation for User Engagement Evaluation
,
2014,
RecSysChallenge '14.
[2]
Simon Dooms,et al.
Recommender systems challenge 2014
,
2014,
RecSys '14.
[3]
Miles Osborne,et al.
RT to Win! Predicting Message Propagation in Twitter
,
2011,
ICWSM.
[4]
W. Bruce Croft,et al.
User oriented tweet ranking: a filtering approach to microblogs
,
2011,
CIKM '11.
[5]
Stephen Kwek,et al.
Applying Support Vector Machines to Imbalanced Datasets
,
2004,
ECML.
[6]
Azadeh Shakery,et al.
Adaptive User Engagement Evaluation via Multi-task Learning
,
2015,
SIGIR.
[7]
Daniele Loiacono,et al.
An analysis of the 2014 RecSys Challenge
,
2014,
RecSysChallenge '14.
[8]
Hamed Zamani,et al.
Multitask Learning for Adaptive Quality Estimation of Automatically Transcribed Utterances
,
2015,
NAACL.
[9]
Philip S. Yu,et al.
Adaptation Regularization: A General Framework for Transfer Learning
,
2014,
IEEE Transactions on Knowledge and Data Engineering.