An Attention-based Deep Network for CTR Prediction

Click-through rate (CTR) prediction is a crucial topic in online advertising system. Early researchers proposed numerous shallow models to analyze this issue, such as logistic regression, factorization machines and Gradient boosting decision tree. In order to improve the model performance furthermore, researchers propose some deep models, such as the factorization-machine supported neural networks, Wide&Deep, DeepFM. Normally, user's historical behavior data contains abundant feature information, but above models lack of modeling and analysis of historical data. To address this problem, the paper proposes a deep CTR prediction model based on attention mechanism and GRU model, which can make use of the users' historical behaviors. The experimental results demonstrate that compared with other popular models, our proposed model can improve the prediction performance by extracting the implied interest features from user historical behaviors.

[1]  Jun Wang,et al.  Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction , 2016, ECIR.

[2]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[3]  Francine Berman,et al.  Social and ethical behavior in the internet of things , 2017, Commun. ACM.

[4]  Yelong Shen,et al.  A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval , 2014, CIKM.

[5]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[6]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[7]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Rómer Rosales,et al.  Simple and Scalable Response Prediction for Display Advertising , 2014, ACM Trans. Intell. Syst. Technol..

[10]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[11]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[12]  Joaquin Quiñonero Candela,et al.  Practical Lessons from Predicting Clicks on Ads at Facebook , 2014, ADKDD'14.

[13]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Martin Wattenberg,et al.  Ad click prediction: a view from the trenches , 2013, KDD.

[16]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[18]  David Lo,et al.  Predicting response in mobile advertising with hierarchical importance-aware factorization machine , 2014, WSDM.

[19]  U. Beier,et al.  Standardization, Evaluation, and Area-Under-Curve Analysis of Human and Murine Treg Suppressive Function. , 2016, Methods in molecular biology.