CHASE: Commonsense-Enriched Advertising on Search Engine with Explicit Knowledge

While online advertising is one of the major sources of income for search engines, pumping up the incomes from business advertisements while ensuring the user experience becomes a challenging but emerging area. Designing high-quality advertisements with persuasive content has been proved as a way to increase revenues through improving the Click-Through Rate (CTR). However, it is difficult to scale up the design of high-quality ads, due to the lack of automation in creativity. In this paper, we present Commonsense-Enriched Advertisement on Search Engine (CHASE) --- a system for the automatic generation of persuasive ads. CHASE adopts a specially designed language model that fuses the keywords, commonsense-related texts, and marketing contents to generate persuasive advertisements. Specifically, the language model has been pre-trained using massive contents of explicit knowledge and fine-tuned with well-constructed quasi-parallel corpora with effective control of the proportion of commonsense in the generated ads and fitness to the ads' keywords. The effectiveness of the proposed method CHASE has been verified by real-world web traffics for search and manual evaluation. In A/B tests, the advertisements generated by CHASE would bring 11.13% CTR improvement. The proposed model has been deployed to cover three advertisement domains (which are kid education, psychological counseling, and beauty e-commerce) at Baidu, the world's largest Chinese search engine, with adding revenue of about 1 million RMB (Chinese Yuan) per day.

[1]  Guillaume Lample,et al.  Multiple-Attribute Text Rewriting , 2018, ICLR.

[2]  Dongyan Zhao,et al.  Low-Resource Knowledge-Grounded Dialogue Generation , 2020, ICLR.

[3]  Yong Yu,et al.  Learning Multi-touch Conversion Attribution with Dual-attention Mechanisms for Online Advertising , 2018, CIKM.

[4]  John Hughes,et al.  Generating Better Search Engine Text Advertisements with Deep Reinforcement Learning , 2019, KDD.

[5]  Ping Li,et al.  MOBIUS: Towards the Next Generation of Query-Ad Matching in Baidu's Sponsored Search , 2019, KDD.

[6]  Robert L. Mercer,et al.  An Estimate of an Upper Bound for the Entropy of English , 1992, CL.

[7]  Jie Tang,et al.  Towards Knowledge-Based Personalized Product Description Generation in E-commerce , 2019, KDD.

[8]  Maxine Eskénazi,et al.  Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders , 2017, ACL.

[9]  Yu Sun,et al.  ERNIE: Enhanced Representation through Knowledge Integration , 2019, ArXiv.

[10]  Ying Li,et al.  Report on the second KDD workshop on data mining for advertising , 2008, SKDD.

[11]  Yu Sun,et al.  Generalizable and Explainable Dialogue Generation via Explicit Action Learning , 2020, FINDINGS.

[12]  Peter Wright,et al.  Persuasion Knowledge , 2022 .

[13]  Foster J. Provost,et al.  Bid optimizing and inventory scoring in targeted online advertising , 2012, KDD.

[14]  Saikat Guha,et al.  Challenges in measuring online advertising systems , 2010, IMC '10.

[15]  Bin Fan,et al.  Knowledge Abstraction Matching for Medical Question Answering , 2019, 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[16]  Vibhanshu Abhishek,et al.  Keyword generation for search engine advertising using semantic similarity between terms , 2007, ICEC.

[17]  Evangelos P. Markatos,et al.  The Cost of Digital Advertisement: Comparing User and Advertiser Views , 2018, WWW.

[18]  Tao Deng,et al.  AiAds: Automated and Intelligent Advertising System for Sponsored Search , 2019, KDD.

[19]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[20]  Kevin Bartz,et al.  Natural language generation for sponsored-search advertisements , 2008, EC '08.

[21]  Jie Zhao,et al.  Riker: Mining Rich Keyword Representations for Interpretable Product Question Answering , 2019, KDD.

[22]  Daniel C. Fain,et al.  Sponsored search: A brief history , 2006 .

[23]  Andrei Z. Broder,et al.  Online expansion of rare queries for sponsored search , 2009, WWW '09.

[24]  Hui Xiong,et al.  Description Generation for Points of Interest , 2021, 2021 IEEE 37th International Conference on Data Engineering (ICDE).

[25]  Ulrich Dolata.  Apple, Amazon, Google, Facebook, Microsoft: Market concentration - competition - innovation strategies , 2017 .

[26]  Bernard J. Jansen,et al.  Sponsored search: an overview of the concept, history, and technology , 2008, Int. J. Electron. Bus..

[27]  Hui Xiong,et al.  A Collaborative Learning Framework to Tag Refinement for Points of Interest , 2019, KDD.

[28]  Jerry Wind,et al.  A Knowledge-Based System for Advertising Design , 1990 .

[29]  Xiaoyan Zhu,et al.  Domain-Constrained Advertising Keyword Generation , 2019, WWW.

[30]  Aaron Flores,et al.  Predicting Different Types of Conversions with Multi-Task Learning in Online Advertising , 2019, KDD.

[31]  Yuanyuan Zhang,et al.  Scalable Query N-Gram Embedding for Improving Matching and Relevance in Sponsored Search , 2018, KDD.

[32]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[33]  Joaquin Quiñonero Candela,et al.  Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine , 2010, ICML.

[34]  Jiliang Tang,et al.  Deep Reinforcement Learning for Search, Recommendation, and Online Advertising: A Survey , 2018 .

[35]  F. Ren,et al.  Multilingual single document keyword extraction for information retrieval , 2005, 2005 International Conference on Natural Language Processing and Knowledge Engineering.

[36]  H. Kemper,et al.  Advertisement and knowledge of tobacco products among Ellisras rural children aged 11 to 18 years: Ellisras Longitudinal study , 2013, BMC Pediatrics.

[37]  Marc'Aurelio Ranzato,et al.  Mixture Models for Diverse Machine Translation: Tricks of the Trade , 2019, ICML.

[38]  Eunice Kim,et al.  Activating persuasion knowledge in native advertising: the influence of cognitive load and disclosure language , 2020, International Journal of Advertising.

[39]  Nader Mohamed,et al.  Statistical techniques for online personalized advertising: a survey , 2012, SAC '12.

[40]  Jason Weston,et al.  Improving Conditioning in Context-Aware Sequence to Sequence Models , 2019, ArXiv.

[41]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[42]  Ke Sun,et al.  Chinese Lexical Analysis with Deep Bi-GRU-CRF Network , 2018, ArXiv.

[43]  Xu Tan,et al.  MASS: Masked Sequence to Sequence Pre-training for Language Generation , 2019, ICML.

[44]  Anthony K. H. Tung,et al.  A Generic Inverted Index Framework for Similarity Search on the GPU , 2016, 2018 IEEE 34th International Conference on Data Engineering (ICDE).

[45]  Mirella Lapata,et al.  Text Generation from Knowledge Graphs with Graph Transformers , 2019, NAACL.