Zapping prediction for online advertisement based on cumulative smile sparse representation

As the amount of online multimedia keeps growing rapidly, study of online user behavior is highly demanded in many applications to provide better user experience and online services. Due to the surging online user population in recent years, more and more online contents are now associated with advertisements. For marketers, online advertising can reach broader audience with reduced cost. Meanwhile, the revenue for many IT companies such as Google mainly comes from advertisement hosting. Therefore, for both marketers and advertisement hosts, effective online advertising is the primary goal. Thus, it is of great interest to predict and prevent the users from zapping, i.e., skipping the advertisement. Reduced zapping would increase the possibility for the user to become potential consumer. Despite its importance, zapping prediction for online advertising has received very limited attention. Since the zapping behavior is related to the user's emotional states, in this paper, we propose to predict zapping from user's facial expressions. Our prediction is non-intrusive, meaning that the user's experience during advertisement watching is not disturbed. Specifically, a robust encoding of the smile responses termed as Cumulative Smile Sparse Representation (CSSR) is extracted from the user's facial expressions. Then, this representation is incorporated in a semi-supervised hypergraph learning framework to predict the moment-to-moment zapping behavior. The hypergraph is able to discover both pairwise and higher order data relationships, leading to more accurate prediction as compared to conventional classifiers. Experiments are conducted on a recently published zapping behavior analysis dataset and state-of-the-art performance is achieved as compared to other competing methods.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Sundar Dorai-Raj,et al.  Empowering Online Advertisements by Empowering Viewers with the Right to Choose , 2012, Journal of Advertising Research.

[3]  Le An,et al.  Zapping Index:Using Smile to Measure Advertisement Zapping Likelihood , 2014, IEEE Transactions on Affective Computing.

[4]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[5]  Marcel Wagner,et al.  Analysis and Improvements of Zapping Times in IPTV Systems , 2009, IEEE Transactions on Broadcasting.

[6]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[7]  Edwin R. Hancock,et al.  Learning Large Scale Class Specific Hyper Graphs for Object Recognition , 2009, 2009 Fifth International Conference on Image and Graphics.

[8]  Jean Ponce,et al.  Learning Graphs to Match , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Hong Shen,et al.  On Identity Disclosure Control for Hypergraph-Based Data Publishing , 2013, IEEE Transactions on Information Forensics and Security.

[10]  K. Poels,et al.  How to Capture the Heart? Reviewing 20 Years of Emotion Measurement in Advertising , 2006 .

[11]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[12]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[13]  Kjell Brunnström,et al.  Perceived quality of channel zapping , 2006, Communication Systems and Networks.

[14]  Lars Bergkvist,et al.  The Flipside of the Sponsorship Coin , 2012, Journal of Advertising Research.

[15]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[16]  R. Pieters,et al.  Emotion-Induced Engagement in Internet Video Advertisements , 2012 .

[17]  Timo Ahonen,et al.  Recognition of blurred faces using Local Phase Quantization , 2008, 2008 19th International Conference on Pattern Recognition.

[18]  Xuelong Li,et al.  Spectral-Spatial Constraint Hyperspectral Image Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[19]  E. Tan,et al.  Predicting advertising effectiveness by facial expressions in response to amusing persuasive stimuli. , 2014 .

[20]  Xuelong Li,et al.  Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search , 2013, IEEE Transactions on Image Processing.

[21]  Paul Gustafson,et al.  Describing the Dynamics of Attention to TV Commercials: A Hierarchical Bayes Analysis of the Time to Zap an Ad , 2007 .

[22]  Raymond Y. K. Lau,et al.  Predicting behavior , 2015, Nature Neuroscience.

[23]  Gang Hua,et al.  Hyperspectral Image Classification Through Bilayer Graph-Based Learning , 2014, IEEE Transactions on Image Processing.

[24]  Deborah A. Small,et al.  The Face of Need: Facial Emotion Expression on Charity Advertisements , 2009 .

[25]  Bir Bhanu,et al.  To skip or not to skip? A dataset of spontaneous affective response of online advertising (SARA) for audience behavior analysis , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[26]  Zhongfei Zhang,et al.  Context-Aware Hypergraph Construction for Robust Spectral Clustering , 2014, 1401.0764.

[27]  R. Pieters,et al.  Why Do Consumers Stop Viewing Television Commercials? Two Experiments on the Influence of Moment-to-Moment Entertainment and Information Value , 2003 .

[28]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..