A Hierarchical Attention Model for CTR Prediction Based on User Interest

The prediction of click-through rate is a challenging problem in the aspect of online advertising. Recently, researchers have proposed deep learning-based models that follow a similar embedding and multilayer perceptron paradigm. Although encouraging successes have been obtained, the importance of capturing the latent user interest behind user behavior data was ignored by most of the methods, which has the potential to effectively learn the feature interactions. In this article, we propose an attentive-deep-interest-based model to fill these gaps. Specifically, we capture the interest sequence in the interest extractor layer, and the auxiliary losses are employed to produce the interest state with deep supervision. First, we use the bidirectional long short-term memory network to model the dependence between behaviors. Next, an interest evolving layer is proposed to extract the interest evolving process that is related to the target. Then, the model learns highly nonlinear interactions of features based on stack autoencoders. An experiment is conducted using four real-world datasets. The experimental results show that the proposed model achieves 1.8% improvement in the Amazon datasets than the existing state-of-the-art models.

[1]  Tie-Yan Liu,et al.  Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks , 2014, AAAI.

[2]  Jianxun Liu,et al.  Functional and Contextual Attention-Based LSTM for Service Recommendation in Mashup Creation , 2019, IEEE Transactions on Parallel and Distributed Systems.

[3]  Wei Xiong,et al.  Stacked Convolutional Denoising Auto-Encoders for Feature Representation , 2017, IEEE Transactions on Cybernetics.

[4]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[5]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[6]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[7]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[8]  Xiaohui Zhao,et al.  A New Approach for Advertising CTR Prediction Based on Deep Neural Network via Attention Mechanism , 2018, Comput. Math. Methods Medicine.

[9]  Martin Wattenberg,et al.  Ad click prediction: a view from the trenches , 2013, KDD.

[10]  Olivier Chapelle,et al.  Modeling delayed feedback in display advertising , 2014, KDD.

[11]  Ke Zhang,et al.  Residual Networks of Residual Networks: Multilevel Residual Networks , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Joaquin Quiñonero Candela,et al.  Practical Lessons from Predicting Clicks on Ads at Facebook , 2014, ADKDD'14.

[14]  Lars Schmidt-Thieme,et al.  Pairwise interaction tensor factorization for personalized tag recommendation , 2010, WSDM '10.

[15]  Bo Wang,et al.  Advertisement Click-Through Rate Prediction Using Multiple Criteria Linear Programming Regression Model , 2013, ITQM.

[16]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[17]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[18]  Nuria Oliver,et al.  Frappe: Understanding the Usage and Perception of Mobile App Recommendations In-The-Wild , 2015, ArXiv.

[19]  Feng Yu,et al.  A Convolutional Click Prediction Model , 2015, CIKM.

[20]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Dong Yu,et al.  Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features , 2016, KDD.

[22]  Guorui Zhou,et al.  Deep Interest Network for Click-Through Rate Prediction , 2017, KDD.

[23]  Jun Wang,et al.  Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction , 2016, ECIR.

[24]  Tat-Seng Chua,et al.  Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks , 2017, IJCAI.

[25]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.

[26]  U. Beier,et al.  Standardization, Evaluation, and Area-Under-Curve Analysis of Human and Murine Treg Suppressive Function. , 2016, Methods in molecular biology.

[27]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[28]  Chong Zhao,et al.  A New Approach for Mobile Advertising Click-Through Rate Estimation Based on Deep Belief Nets , 2017, Comput. Intell. Neurosci..

[29]  Gang Fu,et al.  Deep & Cross Network for Ad Click Predictions , 2017, ADKDD@KDD.

[30]  Yunming Ye,et al.  DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction , 2018, ArXiv.

[31]  Joaquin Quiñonero Candela,et al.  Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine , 2010, ICML.

[32]  Rohit Kumar,et al.  Predicting clicks: CTR estimation of advertisements using Logistic Regression classifier , 2015, 2015 IEEE International Advance Computing Conference (IACC).

[33]  Mingjiang Li,et al.  An improved advertising CTR prediction approach based on the fuzzy deep neural network , 2018, PloS one.

[34]  Xuelong Li,et al.  Describing Video With Attention-Based Bidirectional LSTM , 2019, IEEE Transactions on Cybernetics.

[35]  Rómer Rosales,et al.  Simple and Scalable Response Prediction for Display Advertising , 2014, ACM Trans. Intell. Syst. Technol..

[36]  Matthew Richardson,et al.  Predicting clicks: estimating the click-through rate for new ads , 2007, WWW '07.

[37]  Ilya Trofimov,et al.  Using Neural Networks for Click Prediction of Sponsored Search , 2014, ArXiv.

[38]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[39]  Xiaohui Zhao,et al.  A hierarchical attention model for rating prediction by leveraging user and product reviews , 2019, Neurocomputing.

[40]  Vasudeva Varma,et al.  Predicting the Click-Through Rate for Rare/New Ads , 2022 .

[41]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[42]  Chih-Jen Lin,et al.  Field-aware Factorization Machines for CTR Prediction , 2016, RecSys.