Mining User Interests from Personal Photos

Personal photos are enjoying explosive growth with the popularity of photo-taking devices and social media. The vast amount of online photos largely exhibit users' interests, emotion and opinions. Mining user interests from personal photos can boost a number of utilities, such as advertising, interest based community detection and photo recommendation. In this paper, we study the problem of user interests mining from personal photos. We propose a User Image Latent Space Model to jointly model user interests and image contents. User interests are modeled as latent factors and each user is assumed to have a distribution over them. By inferring the latent factors and users' distributions, we can discover what the users are interested in. We model image contents with a four-level hierarchical structure where the layers correspond to themes, semantic regions, visual words and pixels respectively. Users' latent interests are embedded in the theme layer. Given image contents, users' interests can be discovered by doing posterior inference. We use variational inference to approximate the posteriors of latent variables and learn model parameters. Experiments on 180K Flickr photos demonstrate the effectiveness of our model.

[1]  Xueming Qian,et al.  Recommend Social Network Users Favorite Brands , 2013, PCM.

[2]  Xueming Qian,et al.  Mining user-contributed photos for personalized product recommendation , 2014, Neurocomputing.

[3]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Ryen W. White,et al.  Predicting user interests from contextual information , 2009, SIGIR.

[5]  Alexei A. Efros,et al.  Using Multiple Segmentations to Discover Objects and their Extent in Image Collections , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Bill Triggs,et al.  Region Classification with Markov Field Aspect Models , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[8]  Li Fei-Fei,et al.  Towards total scene understanding: Classification, annotation and segmentation in an automatic framework , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[10]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[11]  Brian D. Davison,et al.  Co-factorization machines: modeling user interests and predicting individual decisions in Twitter , 2013, WSDM.

[12]  Ching-Yung Lin,et al.  On the quality of inferring interests from social neighbors , 2010, KDD.

[13]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[14]  Xing Xie,et al.  Mining user similarity based on location history , 2008, GIS '08.

[15]  Wei-Ying Ma,et al.  Argo: intelligent advertising by mining a user's interest from his photo collections , 2009, KDD Workshop on Data Mining and Audience Intelligence for Advertising.

[16]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories , 2006 .

[17]  Fei-Fei Li,et al.  Spatially Coherent Latent Topic Model for Concurrent Segmentation and Classification of Objects and Scenes , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  Fei-Fei Li,et al.  Image Segmentation with Topic Random Field , 2010, ECCV.

[19]  S. Süsstrunk,et al.  SLIC Superpixels ? , 2010 .

[20]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Alexei A. Efros,et al.  Discovering objects and their location in images , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[22]  Hongyan Liu,et al.  Mining User Interests from Information Sharing Behaviors in Social Media , 2013, PAKDD.

[23]  Feng Qiu,et al.  Automatic identification of user interest for personalized search , 2006, WWW '06.

[24]  Thomas Hofmann,et al.  Probabilistic Latent Semantic Analysis , 1999, UAI.