Intrinsic Image Popularity Assessment

The goal of research in automatic image popularity assessment (IPA) is to develop computational models that can accurately predict the potential of a social image to go viral on the Internet. Here, we aim to single out the contribution of visual content to image popularity, \ie, intrinsic image popularity. Specifically, we first describe a probabilistic method to generate massive popularity-discriminable image pairs, based on which the first large-scale image database for intrinsic IPA (I$^2$PA) is established. We then develop computational models for I$^2$PA based on deep neural networks, optimizing for ranking consistency with millions of popularity-discriminable image pairs. Experiments on Instagram and other social platforms demonstrate that the optimized model performs favorably against existing methods, exhibits reasonable generalizability on different databases, and even surpasses human-level performance on Instagram. In addition, we conduct a psychophysical experiment to analyze various aspects of human behavior in I$^2$PA.

[1]  Jussara M. Almeida,et al.  Using early view patterns to predict the popularity of youtube videos , 2013, WSDM.

[2]  Zhou Wang,et al.  dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs , 2017, IEEE Transactions on Image Processing.

[3]  Raffay Hamid,et al.  What makes an image popular? , 2014, WWW.

[4]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[5]  P. Biernacki,et al.  Snowball Sampling: Problems and Techniques of Chain Referral Sampling , 1981 .

[6]  Joonhwan Lee,et al.  Do private and sexual pictures receive more likes on Instagram? , 2017, 2017 International Conference on Research and Innovation in Information Systems (ICRIIS).

[7]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[8]  Hang Li.  Learning to Rank for Information Retrieval and Natural Language Processing , 2011, Synthesis Lectures on Human Language Technologies.

[9]  L. Thurstone.  A law of comparative judgment. , 1994 .

[10]  Minkyu Kim,et al.  Predicting the Future Popularity of Images on Social Networks , 2016, MISNC.

[11]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[14]  Abhimanyu Dubey,et al.  Modeling Image Virality with Pairwise Spatial Transformer Networks , 2017, ACM Multimedia.

[15]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Joemon M. Jose,et al.  "Nobody comes here anymore, it's too crowded"; Predicting Image Popularity on Flickr , 2014, ICMR.

[17]  Marcel Worring,et al.  Category Specific Post Popularity Prediction , 2018, MMM.

[18]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Devi Parikh,et al.  Understanding image virality , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Przemysław Rokita,et al.  Predicting Popularity of Online Videos Using Support Vector Regression , 2017, IEEE Transactions on Multimedia.

[23]  Yongdong Zhang,et al.  Unfolding Temporal Dynamics: Predicting Social Media Popularity Using Multi-scale Temporal Decomposition , 2016, AAAI.

[24]  David M. Mimno,et al.  Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity , 2017, WWW.

[25]  Jiebo Luo,et al.  How to Become Instagram Famous: Post Popularity Prediction with Dual-Attention , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[26]  Yongdong Zhang,et al.  Sequential Prediction of Social Media Popularity with Deep Temporal Context Networks , 2017, IJCAI.

[27]  Marcel Worring,et al.  Multimodal Popularity Prediction of Brand-related Social Media Posts , 2016, ACM Multimedia.

[28]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[29]  Tomasz Trzcinski,et al.  Pay Attention to Virality: Understanding Popularity of Social Media Videos with the Attention Mechanism , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Chun Wang,et al.  Social Media Prediction Based on Residual Learning and Random Forest , 2017, ACM Multimedia.

[31]  Yongdong Zhang,et al.  Time Matters: Multi-scale Temporalization of Social Media Popularity , 2016, ACM Multimedia.

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Luis E. Ortiz,et al.  Chic or Social: Visual Popularity Analysis in Online Fashion Networks , 2014, ACM Multimedia.

[34]  M Damashek,et al.  Gauging Similarity with n-Grams: Language-Independent Categorization of Text , 1995, Science.

[35]  R. Manmatha,et al.  Predicting retweet count using visual cues , 2013, CIKM.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Eric Gilbert,et al.  Faces engage us: photos with faces attract more likes and comments on Instagram , 2014, CHI.

[38]  Alberto Del Bimbo,et al.  Image Popularity Prediction in Social Media Using Sentiment and Context Features , 2015, ACM Multimedia.

[39]  Melvin Alexander.  Applied Statistics and Probability for Engineers , 1995 .

[40]  Wei Zhang,et al.  User-guided Hierarchical Attention Network for Multi-modal Social Image Popularity Prediction , 2018, WWW.

[41]  Douglas C. Montgomery,et al.  Applied Statistics and Probability for Engineers, Third edition , 1994 .

[42]  Cees Snoek,et al.  Latent Factors of Visual Popularity Prediction , 2015, ICMR.

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.