Bid-Aware Active Learning in Real-Time Bidding for Display Advertising

In Real-time Bidding (RTB) based display advertising, demand side platforms (DSPs) estimate the click-through rate (CTR) of each advertisement impression, and then decide whether and how much to bid based on the information of the user and the advertiser. Typically, when a new campaign is launched, the CTR estimation module of the DSP needs to collect data to train an accurate estimator. The advertiser is charged for each ad impression in display advertising, therefore there is some cost for obtaining each training instance. Thus one crucial task is to actively train an accurate CTR estimator within the constraint of the budget. Traditional active learning algorithms fail to deal with such scenario because (i) acquiring training instances is implemented via performing real-time bidding for the corresponding auctions; (ii) RTB requires the bidding agent to make real-time decisions for sequentially coming bid requests; (iii) cost for each ad impression will be unveiled only after giving the bid price and winning the auction; (iv) training data gathered in post-bid stage has a strong bias towards the won impressions. In this paper, we propose a Bid-aware Active Real-time Bidding (BARB) algorithm to actively choose training instances by setting different bid prices for each ad auction, in order to efficiently train an accurate CTR estimation model within the budget constraint. The empirical study on different campaigns of three real-world datasets with three budget constraints shows the effectiveness of our proposed algorithm.

[1]  Zhihua Zhang,et al.  Learning click models via probit bayesian inference , 2010, CIKM.

[2]  Tie-Yan Liu,et al.  Joint optimization of bid and budget allocation in sponsored search , 2012, KDD.

[3]  Sergei Vassilvitskii,et al.  Bidding for Representative Allocations for Display Advertising , 2009, WINE.

[4]  Shaul Markovitch,et al.  Anytime Induction of Cost-sensitive Trees , 2007, NIPS.

[5]  Weinan Zhang,et al.  Optimal real-time bidding for display advertising , 2014, KDD.

[6]  R. Preston McAfee,et al.  Efficient Ranking in Sponsored Search , 2011, WINE.

[7]  Jun Wang,et al.  Real-Time Bidding Benchmarking with iPinYou Dataset , 2014, ArXiv.

[8]  Jun Wang,et al.  Real-Time Bidding by Reinforcement Learning in Display Advertising , 2017, WSDM.

[9]  Jun Wang,et al.  Functional Bid Landscape Forecasting for Display Advertising , 2016, ECML/PKDD.

[10]  Peter D. Turney.  Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm , 1994, J. Artif. Intell. Res..

[11]  Nikhil R. Devanur,et al.  Real-time bidding algorithms for performance-based display ad allocation , 2011, KDD.

[12]  Wentong Li,et al.  Estimating conversion rate in display advertising from past erformance data , 2012, KDD.

[13]  Shichao Zhang,et al.  Cost-Sensitive Test Strategies , 2006, AAAI.

[14]  Jun Wang,et al.  Sequential selection of correlated ads by POMDPs , 2012, CIKM.

[15]  S. Muthukrishnan,et al.  Ad Exchanges: Research Issues , 2009, WINE.

[16]  Jun Wang,et al.  Product-Based Neural Networks for User Response Prediction , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[17]  Yoav Preund,et al.  Sifting informative examples from a random source. , 1994 .

[18]  Anton Schwaighofer,et al.  Budget Optimization for Sponsored Search: Censored Learning in MDPs , 2012, UAI.

[19]  Sergei Vassilvitskii,et al.  Handling forecast errors while bidding for display advertising , 2012, WWW.

[20]  Jingbo Zhu,et al.  Active Learning for Word Sense Disambiguation with Methods for Addressing the Class Imbalance Problem , 2007, EMNLP.

[21]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[22]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[23]  Luo Si,et al.  Forecasting user visits for online display advertising , 2012, Information Retrieval.

[24]  Brendan Kitts,et al.  A trading agent and simulator for keyword auctions , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[25]  Jun Wang,et al.  Real-time bidding for online advertising: measurement and analysis , 2013, ADKDD '13.

[26]  Eustache Diemert,et al.  Attribution Modeling Increases Efficiency of Bidding in Display Advertising , 2017, ADKDD@KDD.

[27]  Ming-Syan Chen,et al.  Predicting Winning Price in Real Time Bidding with Censored Data , 2015, KDD.

[28]  H. Sebastian Seung,et al.  Information, Prediction, and Query by Committee , 1992, NIPS.

[29]  Wei Li,et al.  Bid landscape forecasting in online ad exchange marketplace , 2011, KDD.

[30]  Jun Wang,et al.  Bid-aware Gradient Descent for Unbiased Learning with Censored Data in Display Advertising , 2016, KDD.

[31]  Joaquin Quiñonero Candela,et al.  Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine , 2010, ICML.

[32]  E. Kaplan,et al.  Nonparametric Estimation from Incomplete Observations , 1958 .

[33]  Foster J. Provost,et al.  Bid optimizing and inventory scoring in targeted online advertising , 2012, KDD.

[34]  Lihong Li,et al.  Unbiased online active learning in data streams , 2011, KDD.

[35]  Hao Wang,et al.  Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising , 2018, CIKM.

[36]  Shlomo Argamon,et al.  Committee-Based Sampling For Training Probabilistic Classi(cid:12)ers , 1995 .

[37]  Jun Wang,et al.  Managing Risk of Bidding in Display Advertising , 2017, WSDM.

[38]  Jun Wang,et al.  User Response Learning for Directly Optimizing Campaign Performance in Display Advertising , 2016, CIKM.

[39]  U. Hahn,et al.  Reducing class imbalance during active learning for named entity annotation , 2009, K-CAP '09.

[40]  S. Ertekin.  CLASS IMBALANCE AND ACTIVE LEARNING , 2013 .

[41]  Michael Ostrovsky,et al.  Reserve Prices in Internet Advertising Auctions: A Field Experiment , 2009, Journal of Political Economy.

[42]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.