Efficient Optimal Selection for Composited Advertising Creatives with Tree Structure

Ad creatives are one of the prominent mediums for online e-commerce advertisements. Ad creatives with enjoyable visual appearance may increase the click-through rate (CTR) of products. Ad creatives are typically handcrafted by advertisers and then delivered to the advertising platforms for advertisement. In recent years, advertising platforms are capable of instantly compositing ad creatives with arbitrarily designated elements of each ingredient, so advertisers are only required to provide basic materials. While facilitating the advertisers, a great number of potential ad creatives can be composited, making it difficult to accurately estimate CTR for them given limited real-time feedback. To this end, we propose an Adaptive and Efficient ad creative Selection (AES) framework based on a tree structure. The tree structure on compositing ingredients enables dynamic programming for efficient ad creative selection on the basis of CTR. Due to limited feedback, the CTR estimator is usually of high variance. Exploration techniques based on Thompson sampling are widely used for reducing variances of the CTR estimator, alleviating feedback sparsity. Based on the tree structure, Thompson sampling is adapted with dynamic programming, leading to efficient exploration for potential ad creatives with the largest CTR. We finally evaluate the proposed algorithm on the synthetic dataset and the real-world dataset. The results show that our approach can outperform competing baselines in terms of convergence rate and overall CTR.

[1]  Yi Liu,et al.  An Efficient Bandit Algorithm for Realtime Multivariate Optimization , 2017, KDD.

[2]  Hongtao Lu,et al.  Deep CTR Prediction in Display Advertising , 2016, ACM Multimedia.

[3]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..

[4]  Shipra Agrawal,et al.  Thompson Sampling for Contextual Bandits with Linear Payoffs , 2012, ICML.

[5]  W. R. Thompson.  ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES , 1933 .

[6]  Liang Tang,et al.  Automatic ad format selection via contextual bandits , 2013, CIKM.

[7]  Lihong Li,et al.  An Empirical Evaluation of Thompson Sampling , 2011, NIPS.

[8]  Yishay Mansour,et al.  Top-$k$ Combinatorial Bandits with Full-Bandit Feedback , 2020, ALT.

[9]  Xiaoli Z. Fern,et al.  The impact of visual appearance on user response in online display advertising , 2012, WWW.

[10]  Yong Li,et al.  Image Feature Learning for Cold Start Problem in Display Advertising , 2015, IJCAI.

[11]  Wei Li,et al.  Exploitation and exploration in a performance based contextual advertising system , 2010, KDD.

[12]  Wei Chu,et al.  Contextual Bandits with Linear Payoff Functions , 2011, AISTATS.

[13]  Choon Hui Teo,et al.  Adaptive, Personalized Diversity for Visual Discovery , 2016, RecSys.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Xiwei Zhao,et al.  Category-Specific CNN for Visual-aware CTR Prediction at JD.com , 2020, KDD.

[16]  Qing Wang,et al.  Online IT Ticket Automation Recommendation Using Hierarchical Multi-armed Bandit Algorithms , 2018, SDM.

[17]  Joaquin Quiñonero Candela,et al.  Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine , 2010, ICML.

[18]  Rémi Munos,et al.  Open Loop Optimistic Planning , 2010, COLT.

[19]  Wei Chen,et al.  Combinatorial Multi-Armed Bandit: General Framework and Applications , 2013, ICML.

[20]  Klemens Böhm,et al.  Scaling Multi-Armed Bandit Algorithms , 2019, KDD.

[21]  Yue Wang,et al.  Beyond Ranking: Optimizing Whole-Page Presentation , 2016, WSDM.

[22]  Yisong Yue,et al.  Hierarchical Exploration for Accelerating Contextual Bandits , 2012, ICML.

[23]  Tingfa Xu,et al.  LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators , 2019, ICLR.

[24]  Yang Zhou,et al.  Multimedia features for click prediction of new ads in display advertising , 2012, KDD.

[25]  Nicolò Cesa-Bianchi,et al.  Combinatorial Bandits , 2012, COLT.

[26]  Xian-Sheng Hua.  Challenges and Practices of Large Scale Visual Intelligence in the Real-World , 2018, ACM Multimedia.

[27]  P. Read Montague,et al.  Reinforcement Learning: An Introduction, by Sutton, R.S. and Barto, A.G. , 1999, Trends in Cognitive Sciences.