E-commerce in Your Inbox: Product Recommendations at Scale

In recent years online advertising has become increasingly ubiquitous and effective. Advertisements shown to visitors fund sites and apps that publish digital content, manage social networks, and operate e-mail services. Given such large variety of internet resources, determining an appropriate type of advertising for a given platform has become critical to financial success. Native advertisements, namely ads that are similar in look and feel to content, have had great success in news and social feeds. However, to date there has not been a winning formula for ads in e-mail clients. In this paper we describe a system that leverages user purchase history determined from e-mail receipts to deliver highly personalized product ads to Yahoo Mail users. We propose to use a novel neural language-based algorithm specifically tailored for delivering effective product recommendations, which was evaluated against baselines that included showing popular products and products predicted based on cooccurrence. We conducted rigorous offline testing using a large-scale product purchase data set, covering purchases of more than 29 million users from 172 e-commerce websites. Ads in the form of product recommendations were successfully tested on online traffic, where we observed a steady 9% lift in click-through rates over other ad formats in mail, as well as comparable lift in conversion rates. Following successful tests, the system was launched into production during the holiday season of 2014.

[1]  Ruslan Salakhutdinov,et al.  A Multiplicative Model for Learning Distributed Text-Based Attribute Representations , 2014, NIPS.

[2]  Yongzheng Zhang,et al.  Predicting purchase behaviors from social media , 2013, WWW.

[3]  Daniel Gooch,et al.  Communications of the ACM , 2011, XRDS.

[4]  Nemanja Djuric,et al.  Hidden Conditional Random Fields with Distributed User Embeddings for Ad Targeting , 2014 .

[5]  Ruslan Salakhutdinov,et al.  Multimodal Neural Language Models , 2014, ICML.

[6]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[7]  W. Bruce Croft,et al.  Relevance-Based Language Models , 2001, SIGIR '01.

[8]  Slobodan Vucetic,et al.  Generating ad targeting rules using sparse principal component analysis with constraints , 2014, WWW '14 Companion.

[9]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[10]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[11]  Rob Hall,et al.  Style in the long tail: discovering unique interests with latent variable models in large scale social E-commerce , 2014, KDD.

[12]  Chang-Hoan Cho,et al.  WHY DO PEOPLE AVOID ADVERTISING ON THE INTERNET? , 2004 .

[13]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[14]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[15]  Nemanja Djuric,et al.  Search Retargeting using Directed Query Embeddings , 2015, WWW.

[16]  Georgiana Dinu,et al.  Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors , 2014, ACL.

[17]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[18]  Yoshua Bengio,et al.  Neural Probabilistic Language Models , 2006 .

[19]  David Essex,et al.  Matchmaker, matchmaker , 2009, CACM.

[20]  Tolga Könik,et al.  Recommending similar items in large-scale online marketplaces , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[21]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[22]  John G. Lynch,et al.  Interactive Home Shopping: Consumer, Retailer, and Manufacturer Incentives to Participate in Electronic Marketplaces , 1997 .

[23]  Alexander J. Smola,et al.  Scalable distributed inference of dynamic user interests for behavioral targeting , 2011, KDD.

[24]  Yuexin Wu,et al.  We know what you want to buy: a demographic-based system for product recommendation on microblogs , 2014, KDD.

[25]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[26]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[27]  Hao Wu,et al.  Hierarchical Neural Language Models for Joint Representation of Streaming Documents and their Content , 2015, WWW.

[28]  Yoelle Maarek,et al.  How Many Folders Do You Really Need?: Classifying Email into a Handful of Categories , 2014, CIKM.