Understanding Consumer Journey using Attention based Recurrent Neural Networks

Paths of online users towards a purchase event (conversion) can be very complex, and guiding them through their journey is an integral part of online advertising. Studies in marketing indicate that a conversion event is typically preceded by one or more purchase funnel stages, viz., unaware, aware, interest, consideration, and intent. Intuitively, some online activities, including web searches, site visits and ad interactions, can serve as markers for the user's funnel stage. Identifying such markers can potentially refine conversion prediction, guide the design of ad creatives (text and images), and lead to higher ad effectiveness. We explore this hypothesis through a set of experiments designed for two tasks: (i) conversion prediction given a user's activity trail, and (ii) funnel stage specific targeting and creatives. To address challenges in the two tasks, we propose an attention based recurrent neural network (RNN) which ingests a user activity trail, and predicts the user's conversion probability along with attention weights for each activity (analogous to its position in the funnel). Specifically, we propose novel attention mechanisms, which maintain a global weight for each activity across all user trails, and also indicate the activity's funnel stage. Use of the proposed attention mechanisms for the first task of conversion prediction shows significant AUC lifts of 0.9% on a public dataset (RecSys 2015 challenge), and up to 3.6% on three proprietary datasets from a major advertising platform (Yahoo Gemini). To address the second task, the activity weights from the proposed mechanisms are used to automatically assign users to funnel stages via a scalable scoring method. Offline evaluation shows that such activity weights are more aligned with editorially tagged activity-funnel stages compared to weights from existing attention mechanisms and simpler conversion models like logistic regression. In addition, results of online ad campaigns in Yahoo Gemini with funnel specific user targeting and ad creatives show strong performance lifts further validating the connection across online activities, purchase funnel stages, stage-specific custom creatives, and conversions.

[1]  Zoran Obradovic,et al.  Modeling Mobile User Actions for Purchase Recommendation using Deep Memory Networks , 2018, SIGIR.

[2]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[3]  Vibhanshu Abhishek,et al.  Media Exposure through the Funnel: A Model of Multi-Stage Attribution , 2012 .

[4]  G. Jenks.  The Data Model Concept in Statistical Mapping , 1967 .

[5]  Martin Wattenberg,et al.  Ad click prediction: a view from the trenches , 2013, KDD.

[6]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[7]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[9]  Yanwei Cui,et al.  Modelling customer online behaviours with neural networks: applications to conversion prediction and advertising retargeting , 2018, ArXiv.

[10]  Yong Yu,et al.  Learning Multi-touch Conversion Attribution with Dual-attention Mechanisms for Online Advertising , 2018, CIKM.

[11]  Jiawei Han,et al.  I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application , 2018, KDD.

[12]  Param Vir Singh,et al.  Got Annoyed? Examining the Advertising Effectiveness and Annoyance Dynamics , 2017, ICIS.

[13]  Dong Yu,et al.  Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features , 2016, KDD.

[14]  Guorui Zhou,et al.  Deep Interest Network for Click-Through Rate Prediction , 2017, KDD.

[15]  Bernard J. Jansen,et al.  BIDDING ON THE BUYING FUNNEL FOR SPONSORED SEARCH AND KEYWORD ADVERTISING , 2011 .

[16]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[17]  Ravi Kant,et al.  A Large Scale Prediction Engine for App Install Clicks and Conversions , 2017, CIKM.

[18]  Mirella Lapata,et al.  Long Short-Term Memory-Networks for Machine Reading , 2016, EMNLP.

[19]  Tie-Yan Liu,et al.  Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks , 2014, AAAI.

[20]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[21]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[22]  Zhongfei Zhang,et al.  DeepIntent: Learning Attentions for Online Advertising with Recurrent Neural Networks , 2016, KDD.

[23]  Zoran Obradovic,et al.  Deeply supervised model for click-through rate prediction in sponsored search , 2019, Data Mining and Knowledge Discovery.