Uncovering Bias in Ad Feedback Data Analyses & Applications✱

Electronic publishers and other web-companies are starting to collect user feedback on ads with the aim of using this signal to maintain the quality of ads shown on their sites. However, users are not randomly sampled to provide feedback on ads, but targeted. Furthermore some users who provide feedback may be prone to dislike ads more than the general user. This raises questions about the reliability of ad feedback as a signal for measuring ad quality and whether it can be used in ad ranking. In this paper we start by gaining insights to such signals by analyzing the feedback event logs attributed to users of a popular mobile news app. We then propose a model to reduce potential biases in ad feedback data. Finally, we conclude by comparing the effectiveness of reducing the bias in ad feedback data using existing ad ranking methods along with a new and novel approach we propose that takes revenue considerations into account.

[1]  Sergei Vassilvitskii,et al.  Advertising in a stream , 2014, WWW.

[2]  Liron Levin,et al.  OFF-set: one-pass factorization of feature sets for online recommendation in persistent cold start settings , 2013, RecSys.

[3]  Hila Becker,et al.  What happens after an ad click?: quantifying the impact of landing pages in web advertising , 2009, CIKM.

[4]  Dustin Hillard,et al.  A predictive model for advertiser value-per-click in sponsored search , 2013, WWW.

[5]  Tie-Yan Liu,et al.  Psychological advertising: exploring user psychology for click prediction in sponsored search , 2013, KDD.

[6]  Fabrizio Silvestri,et al.  Promoting Positive Post-Click Experience for In-Stream Yahoo Gemini Users , 2015, KDD.

[7]  D. Sculley,et al.  Predicting bounce rates in sponsored search advertisements , 2009, KDD.

[8]  David Lo,et al.  Predicting response in mobile advertising with hierarchical importance-aware factorization machine , 2014, WSDM.

[9]  F. Vella.  Estimating Models with Sample Selection Bias: A Survey , 1998 .

[10]  Xiaoli Z. Fern,et al.  Visual appearance of display ads and its effect on click through rate , 2012, CIKM.

[11]  Silvia Gabrielli,et al.  A Review of Online Advertising Effects on the User Experience , 2010, Int. J. Hum. Comput. Interact..

[12]  Ricardo Baeza-Yates,et al.  Scalable Semantic Matching of Queries to Ads in Sponsored Search Advertising , 2016, SIGIR.

[13]  Galit Shmueli,et al.  To Explain or To Predict? , 2010, 1101.0891.

[14]  Diane Tang,et al.  Focusing on the Long-term: It's Good for Users and Business , 2015, KDD.

[15]  R. Preston McAfee,et al.  The cost of annoying ads , 2013, WWW '13.

[16]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[17]  Fabrizio Silvestri,et al.  Improving Post-Click User Engagement on Native Ads via Survival Analysis , 2016, WWW.

[18]  Andrei Z. Broder,et al.  To swing or not to swing: learning when (not) to advertise , 2008, CIKM '08.

[19]  Brian D. Davison,et al.  Estimating ad group performance in sponsored search , 2014, WSDM.

[20]  S. Athey,et al.  A Structural Model of Sponsored Search Advertising Auctions , 2011 .

[21]  Charles L. A. Clarke,et al.  Modeling browsing behavior for click analysis in sponsored search , 2012, CIKM '12.

[22]  Yang Zhou,et al.  Multimedia features for click prediction of new ads in display advertising , 2012, KDD.

[23]  Brendan Lucier,et al.  User satisfaction in competitive sponsored search , 2013, WWW.

[24]  Ke Zhou,et al.  Predicting Pre-click Quality for Native Advertisements , 2016, WWW.

[25]  Rómer Rosales,et al.  Post-click conversion modeling and analysis for non-guaranteed delivery display advertising , 2012, WSDM '12.

[26]  Dragomir Yankov,et al.  Categorization of display ads using image and landing page features , 2011, LDMTA '11.

[27]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[28]  Yejin Choi,et al.  Using landing pages for sponsored search ad selection , 2010, WWW '10.

[29]  Henriette Cramer,et al.  Effects of Ad Quality & Content-Relevance on Perceived Content Quality , 2015, CHI.