Improvement of natural image search engines results by emotional filtering

With the Internet 2.0 era, managing user emotions is a problem that more and more actors are interested in. Historically, the first notions of emotion sharing were expressed and defined with emoticons. They allowed users to show their emotional status to others in an impersonal and emotionless digital world. Now, in the Internet of social media, every day users share lots of content with each other on Facebook, Twitter, Google+ and so on. Several new popular web sites like FlickR, Picassa, Pinterest, Instagram or DeviantArt are now specifically based on sharing image content as well as personal emotional status. This kind of information is economically very valuable as it can for instance help commercial companies sell more efficiently. In fact, with this king of emotional information, business can made where companies will better target their customers needs, and/or even sell them more products. Research has been and is still interested in the mining of emotional information from user data since then. In this paper, we focus on the impact of emotions from images that have been collected from search image engines. More specifically our proposition is the creation of a filtering layer applied on the results of such image search engines. Our peculiarity relies in the fact that it is the first attempt from our knowledge to filter image search engines results with an emotional filtering approach.

[1]  Nicu Sebe,et al.  Emotional valence categorization using holistic image features , 2008, 2008 15th IEEE International Conference on Image Processing.

[2]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[3]  Reiner Lenz,et al.  Emotion related structures in large image databases , 2010, CIVR '10.

[4]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[5]  Emmanuel Dellandréa,et al.  Evaluation of Features and Combination Approaches for the Classification of Emotional Semantics in Images , 2011, VISAPP.

[6]  Christine Fernandez-Maloigne,et al.  Can Salient Interest Regions Resume Emotional Impact of an Image? , 2013, CAIP.

[7]  L. Ou,et al.  A study of colour emotion and colour preference. Part II: Colour emotions for two‐colour combinations , 2004 .

[8]  Reiner Lenz,et al.  Color Based Bags-of-Emotions , 2009, CAIP.

[9]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[10]  Kathrin Knautz,et al.  MEMOSE: search engine for emotions in multimedia documents , 2010, SIGIR.

[11]  David Fonseca,et al.  An image-centred "search and indexation system" based in user's data and perceived emotion , 2008, HCC '08.

[12]  Marie L. Smith,et al.  Transmission of Facial Expressions of Emotion Co-Evolved with Their Efficient Decoding in the Brain: Behavioral and Brain Evidence , 2009, PloS one.

[13]  P. Ekman,et al.  Facial Expressions of Emotion , 1979 .

[14]  Wei-Ning Wang,et al.  Image emotional semantic query based on color semantic description , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[15]  Jonathon S. Hare,et al.  Bridging the Semantic Gap in Multimedia Information Retrieval: Top-down and Bottom-up approaches , 2006 .

[16]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[18]  M. Bradley,et al.  Emotion and motivation II: sex differences in picture processing. , 2001, Emotion.

[19]  L. Ou,et al.  A study of colour emotion and colour preference. Part I: Colour emotions for single colours , 2004 .

[20]  Jonathon Read,et al.  Using Emoticons to Reduce Dependency in Machine Learning Techniques for Sentiment Classification , 2005, ACL.

[21]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[22]  Kenji Araki,et al.  Emoticon recommendation system for effective communication , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[23]  P. Bodrogi,et al.  Color preference of aged observers compared to young observers , 2008 .

[24]  Robert Marti,et al.  Which is the best way to organize/classify images by content? , 2007, Image Vis. Comput..

[25]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Simon Rigoulot,et al.  Impact comportemental et électrophysiologique de l'information émotionnelle en vision périphérique , 2008 .

[27]  Mingjing Li,et al.  Color texture moments for content-based image retrieval , 2002, Proceedings. International Conference on Image Processing.

[28]  Gijs Huisman,et al.  LEMtool: measuring emotions in visual interfaces , 2013, CHI.

[29]  Hsin-Hsi Chen,et al.  Emotion Classification of Online News Articles from the Reader's Perspective , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[30]  Christine Fernandez-Maloigne,et al.  Extraction of emotional impact in colour images , 2012, CGIV.

[31]  Kah Phooi Seng,et al.  A new approach of audio emotion recognition , 2014, Expert Syst. Appl..

[32]  Jonathan Harris,et al.  We feel fine and searching the emotional web , 2011, WSDM '11.

[33]  Benoit Huet,et al.  Toward emotion indexing of multimedia excerpts , 2008, 2008 International Workshop on Content-Based Multimedia Indexing.

[34]  Aly A. Farag,et al.  CSIFT: A SIFT Descriptor with Color Invariant Characteristics , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[35]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[36]  Bernard Rimé.  Émotion et mémoire: la rémanence des expériences émotionnefles , 2014 .

[37]  L. Ou,et al.  A study of colour emotion and colour preference. Part III: Colour preference modeling , 2004 .

[38]  Huan Liu,et al.  Unsupervised sentiment analysis with emotional signals , 2013, WWW.

[39]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[40]  Matthieu Perreira Da Silva,et al.  Implementation and evaluation of a computational model of attention for computer vision , 2013 .

[41]  Marcel P. Lucassen,et al.  Adding texture to color: quantitative analysis of color emotions , 2010, CGIV/MCS.

[42]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[43]  Mike Graham,et al.  Extracting information about emotions in films , 2003, ACM Multimedia.

[44]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[45]  Tao Zhang,et al.  Image Emotional Classification Based on Color Semantic Description , 2008, ADMA.

[46]  Luc Van Gool,et al.  Moment invariants for recognition under changing viewpoint and illumination , 2004, Comput. Vis. Image Underst..

[47]  H. Conte,et al.  Circumplex models of personality and emotions , 1997 .

[48]  Wei-Ying Ma,et al.  EnjoyPhoto: a vertical image search engine for enjoying high-quality photos , 2006, MM '06.

[49]  C. Boyatzis,et al.  Children's emotional associations with colors. , 1994, The Journal of genetic psychology.

[50]  Xian-Sheng Hua,et al.  The role of attractiveness in web image search , 2011, ACM Multimedia.