Sentic Album: Content-, Concept-, and Context-Based Online Personal Photo Management System

The world of online personal photo management has come a long way in the past few years, but today, there are still huge gaps in annotating, organizing, and retrieving online pictures in such a way that they can be easily queried and visualized. Existing content-based image retrieval systems apply statistics, pattern recognition, signal processing, and computer vision techniques but these are still too weak to ‘bridge the semantic gap’ between the low-level data representation and the high-level concepts the user associates with images. Image meta search engines, on the other hand, rely on tags associated with online pictures but results are often too inaccurate since they mainly depend on keyword-based rather than concept-based algorithms. Sentic Album is a novel content-, concept-, and context-based online personal photo management system that exploits both data and metadata of online personal pictures to intelligently annotate, organize, and retrieve them. Many salient features of pictures, in fact, are only noticeable in the viewer’s mind, and the cognitive ability to grasp such features is a key aspect for accordingly analyzing and classifying personal photos. To this end, Sentic Album exploits not just colors and texture of online images (content), but also the cognitive and affective information associated with their metadata (concept), and their relative timestamp, geolocation, and user interaction metadata (context).

[1]  Boris Motik,et al.  Query Answering for OWL-DL with Rules , 2004, SEMWEB.

[2]  P. Wilson,et al.  The Nature of Emotions , 2012 .

[3]  Kriengkrai Porkaew,et al.  Query refinement for multimedia similarity retrieval in MARS , 1999, MULTIMEDIA '99.

[4]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[5]  Xinbo Gao,et al.  No-reference image quality assessment in contourlet domain , 2010, Neurocomputing.

[6]  Erik Cambria,et al.  Sentic Medoids: Organizing Affective Common Sense Knowledge in a Multi-Dimensional Vector Space , 2011, ISNN.

[7]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[8]  Elizabeth F. Loftus,et al.  Remembering emotional events: the fate of detailed information , 1991 .

[9]  Judith Redi,et al.  Color Distribution Information for the Reduced-Reference Assessment of Perceived Image Quality , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  M. Minsky.  The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind , 2006 .

[11]  Bing Liu,et al.  Mining Opinion Features in Customer Reviews , 2004, AAAI.

[12]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[13]  Henry Lieberman,et al.  Digital Intuition: Applying Common Sense Using Dimensionality Reduction , 2009, IEEE Intelligent Systems.

[14]  S LewMichael,et al.  Content-based multimedia information retrieval , 2006 .

[15]  Luis Mateus Rocha,et al.  Singular value decomposition and principal component analysis , 2003 .

[16]  Joemon M. Jose,et al.  EGO: A personalized multimedia management and retrieval tool , 2006, Int. J. Intell. Syst..

[17]  Wei-Ying Ma,et al.  IGroup: web image search results clustering , 2006, MM '06.

[18]  Erik Cambria,et al.  The Hourglass of Emotions , 2011, COST 2102 Training School.

[19]  Erik Cambria,et al.  Sentic Computing for social media marketing , 2012, Multimedia Tools and Applications.

[20]  Marco Grassi.  Developing HEO Human Emotions Ontology , 2009, COST 2101/2102 Conference.

[21]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[22]  J. Itten.  The art of color : the subjective experience and objective rationale of color , 1973 .

[23]  Joemon M. Jose,et al.  An adaptive technique for content-based image retrieval , 2006, Multimedia Tools and Applications.

[24]  J. Gross,et al.  Emotion Regulation in Romantic Relationships: The Cognitive Consequences of Concealing Feelings , 2003 .

[25]  Harald Merckelbach,et al.  The impact of anxiety on memory for details in spider phobics , 1997 .

[26]  Thomas S. Huang,et al.  ImageGrouper: Search, Annotate and Organize Images by Groups , 2002, VISUAL.

[27]  Nicu Sebe,et al.  Evaluation of Salient Point Techniques , 2002, CIVR.

[28]  Weisi Lin,et al.  Objective Image Quality Assessment Based on Support Vector Regression , 2010, IEEE Transactions on Neural Networks.

[29]  George Lakoff,et al.  Women, Fire, and Dangerous Things , 1987 .

[30]  Suh-Yin Lee,et al.  Recent Advances in Visual Information Systems , 2002, Lecture Notes in Computer Science.

[31]  Jana Urban,et al.  AN ADAPTIVE APPROACH TOWARDS CONTENT-BASED IMAGE RETRIEVAL , 2003 .

[32]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[33]  Erik Cambria,et al.  Sentic Activation: A Two-Level Affective Common Sense Reasoning Framework , 2012, AAAI.

[34]  Daniel Reisberg,et al.  Remembering emotional events , 1992, Memory & cognition.

[35]  Qianhua He,et al.  A survey on emotional semantic image retrieval , 2008, 2008 15th IEEE International Conference on Image Processing.

[36]  Daniel Reisberg,et al.  Memory for thematically arousing events , 2004, Memory & cognition.

[37]  Erik Cambria,et al.  Sentic PROMs: Application of sentic computing to the development of a novel unified framework for measuring health-care quality , 2012, Expert Syst. Appl..

[38]  Shih-Fu Chang,et al.  Image and video search engine for the World Wide Web , 1997, Electronic Imaging.

[39]  Anna Esposito,et al.  Biometric ID Management and Multimodal Communication, Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Madrid, Spain, September 16-18, 2009. Proceedings , 2009, COST 2101/2102 Conference.

[40]  Alan F. Smeaton,et al.  MediAssist: Using Content-Based Analysis and Context to Manage Personal Photo Collections , 2006, CIVR.

[41]  Werner Dubitzky,et al.  A Practical Approach to Microarray Data Analysis , 2003, Springer US.

[42]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[43]  Alberto Del Bimbo,et al.  Proceedings of the international conference on Multimedia , 2010 .

[44]  Daniel Reisberg,et al.  Memory and emotion. , 2004 .

[45]  A. Damasio.  Descartes’ Error. Emotion, Reason and the Human Brain. New York (Grosset/Putnam) 1994. , 1994 .

[46]  Huajun Chen,et al.  The Semantic Web , 2011, Lecture Notes in Computer Science.

[47]  Nadia Bianchi-Berthouze,et al.  K-DIME: An Affective Image Filtering System , 2003, IEEE Multim..

[48]  Henry Lieberman,et al.  Aria: an agent for annotating and retrieving images , 2001, Computer.

[49]  Erik Cambria,et al.  Sentic Computing for patient centered applications , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[50]  Aya Soffer,et al.  PicASHOW: pictorial authority search by hyperlinks on the Web , 2001, WWW '01.

[51]  Catherine Havasi,et al.  Automated Color Selection Using Semantic Knowledge , 2010, AAAI Fall Symposium: Commonsense Knowledge.

[52]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[53]  Michael David Walsh,et al.  Hong Kong 2001 , 2001 .

[54]  P. Valdez,et al.  Effects of color on emotions. , 1994, Journal of experimental psychology. General.

[55]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  James A. Hendler,et al.  Agents and the Semantic Web , 2001, IEEE Intell. Syst..

[57]  Michael J. Swain,et al.  WebSeer: An Image Search Engine for the World Wide Web , 1996 .

[58]  Karin Coifman,et al.  The Importance of Being Flexible , 2004, Psychological science.

[59]  Rosalind W. Picard.  Affective Computing , 1997 .

[60]  Henry Lieberman,et al.  Out of context: Computer systems that adapt to, and learn from, context , 2000, IBM Syst. J..

[61]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[62]  Swapna Somasundaran,et al.  Discourse Level Opinion Interpretation , 2008, COLING.

[63]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[64]  Catherine Havasi,et al.  ConceptNet 3 : a Flexible , Multilingual Semantic Network for Common Sense Knowledge , 2007 .

[65]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[66]  Hae-Sang Park,et al.  A simple and fast algorithm for K-medoids clustering , 2009, Expert Syst. Appl..

[67]  Judith Redi,et al.  Circular-ELM for the reduced-reference assessment of perceived image quality , 2013, Neurocomputing.

[68]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[69]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[70]  Erik Cambria,et al.  Sentic Computing: Techniques, Tools, and Applications , 2012 .

[71]  C. Elliott.  The affective reasoner: a process model of emotions in a multi-agent system , 1992 .

[72]  E. Cambria,et al.  AffectiveSpace: Blending Common Sense and Affective Knowledge to Perform Emotive Reasoning , 2009 .

[73]  Henry Lieberman,et al.  Intelligent assistance for conversational storytelling using story patterns , 2011, IUI '11.

[74]  Amarnath Gupta,et al.  Virage image search engine: an open framework for image management , 1996, Electronic Imaging.

[75]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[76]  A. Hanjalic,et al.  Extracting moods from pictures and sounds: towards truly personalized TV , 2006, IEEE Signal Processing Magazine.

[77]  James Ze Wang,et al.  ACQUINE: aesthetic quality inference engine - real-time automatic rating of photo aesthetics , 2010, MIR '10.

[78]  Erik Cambria,et al.  Sentic Panalogy: Swapping Affective Common Sense Reasoning Strategies and Foci , 2012, CogSci.

[79]  Ernest A. Edmonds,et al.  Using Memory for Events in the Design of Personal Filing Systems , 1992, Int. J. Man Mach. Stud..

[80]  Sean A. Spence,et al.  Descartes' Error: Emotion, Reason and the Human Brain , 1995 .

[81]  Jonathan W. Schooler,et al.  Memory for emotional events , 2000 .