A Context Centric Approach for Semantic Image Annotation and Retrieval

In this preliminary research, we discuss techniques to improve the quality of image retrieval and image management with the help of context information over the web. Our hypothesis is that leveraging the semantic annotated contextual metadata of the image would yield the relevant search results and facilitate building a consistent, unambiguous image knowledge base. Inferencing capability of semantic technology is very useful to deduce the new metadata from the existing metadata about images. We discuss the three different aspects of image context such as spatial, temporal and social context. Spatial and temporal context of the image can easily be gathered by taking advantage of modern digital camera technology. In order to acquire social contextual metadata, we benefit from social network sites.

[1]  Jiebo Luo,et al.  Using Geotags to Derive Rich Tag-Clouds for Image Annotation , 2011, Social Media Modeling and Computing.

[2]  Vincent Y. Shen,et al.  User identification across multiple social networks , 2009, 2009 First International Conference on Networked Digital Technologies.

[3]  Bettina Berendt,et al.  Finding Your Way through Blogspace: Using Semantics for Cross-Domain Blog Analysis , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[4]  Gareth J. F. Jones,et al.  Context-aware Retrieval: Exploring a New Environment for Information Retrieval and Information Filtering , 2001, Personal and Ubiquitous Computing.

[5]  Alessandro Acquisti,et al.  Information revelation and privacy in online social networks , 2005, WPES '05.

[6]  G. Walsh,et al.  Why Online Identity Theft Poses a Major Threat to E-Business , 2011 .

[7]  L. Freeman Centrality in social networks conceptual clarification , 1978 .

[8]  Benjamin Nowack CONFOTO: Browsing and annotating conference photos on the Semantic web , 2006, J. Web Semant..

[9]  Huajun Chen,et al.  The Semantic Web , 2011, Lecture Notes in Computer Science.

[10]  Thomas R. Gruber,et al.  A translation approach to portable ontology specifications , 1993, Knowl. Acquis..

[11]  Antonio Hernando,et al.  Collaborative filtering adapted to recommender systems of e-learning , 2009, Knowl. Based Syst..

[12]  Bruno Martins,et al.  Tag recommendation for georeferenced photos , 2011, LBSN '11.

[13]  Ngoc Thanh Nguyen,et al.  Centrality measurement on semantically multiplex social networks: divide-and-conquer approach , 2007, Int. J. Intell. Inf. Database Syst..

[14]  U. Brandes A faster algorithm for betweenness centrality , 2001 .

[15]  Amit P. Sheth,et al.  Semantics-Empowered Social Computing , 2009, IEEE Internet Computing.

[16]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[17]  Enrique Herrera-Viedma,et al.  Dealing with incomplete information in a fuzzy linguistic recommender system to disseminate information in university digital libraries , 2010, Knowl. Based Syst..

[18]  Hugo Liu InterestMap : Harvesting Social Network Profiles for Recommendations , 2004 .

[19]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[20]  Michael Baur,et al.  visone - Software for the Analysis and Visualization of Social Networks , 2008 .

[21]  Reading the world’s mind: Political, socioeconomic and cultural approaches to understanding worldwide Internet search queries , 2011 .

[22]  Panagiotis Symeonidis,et al.  Collaborative recommender systems: Combining effectiveness and efficiency , 2008, Expert Syst. Appl..

[23]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[24]  Olivier Corby,et al.  Analysis of a Real Online Social Network Using Semantic Web Frameworks , 2009, SEMWEB.

[25]  Adrian Popescu,et al.  MonuAnno: automatic annotation of georeferenced landmarks images , 2009, CIVR '09.

[26]  Amit P. Sheth,et al.  Semantic analytics on social networks: experiences in addressing the problem of conflict of interest detection , 2006, WWW '06.

[27]  Bernhard Debatin,et al.  Facebook and Online Privacy: Attitudes, Behaviors, and Unintended Consequences , 2009, J. Comput. Mediat. Commun..

[28]  James Ze Wang,et al.  Learning the consensus on visual quality for next-generation image management , 2007, ACM Multimedia.

[29]  Nixon,et al.  Feature Extraction & Image Processing , 2008 .

[30]  Changhu Wang,et al.  Scalable search-based image annotation of personal images , 2006, MIR '06.

[31]  A. T. Schreiber,et al.  Semantic Annotation of Image Collections , 2003 .

[32]  Gerhard Weikum,et al.  Exploiting social relations for query expansion and result ranking , 2008, 2008 IEEE 24th International Conference on Data Engineering Workshop.

[33]  Jesús Bobadilla,et al.  A new collaborative filtering metric that improves the behavior of recommender systems , 2010, Knowl. Based Syst..

[34]  Qi Gao,et al.  Semantic Enrichment of Twitter Posts for User Profile Construction on the Social Web , 2011, ESWC.

[35]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[36]  Kyong Joo Oh,et al.  The collaborative filtering recommendation based on SOM cluster-indexing CBR , 2003, Expert Syst. Appl..

[37]  Paul Ormerod,et al.  Why Most Things Fail: Evolution, Extinction and Economics , 2005 .

[38]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  B. S. Manjunath,et al.  Spirittagger: a geo-aware tag suggestion tool mined from flickr , 2008, MIR '08.

[40]  Bruce Krulwich,et al.  LIFESTYLE FINDER: Intelligent User Profiling Using Large-Scale Demographic Data , 1997, AI Mag..

[41]  Danah Boyd,et al.  Friendster and publicly articulated social networking , 2004, CHI EA '04.

[42]  Michael J. Muller,et al.  Motivations for social networking at work , 2008, CSCW.

[43]  M. Newman,et al.  The structure of scientific collaboration networks. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[44]  Eileen Wood,et al.  All about me: Disclosure in online social networking profiles: The case of FACEBOOK , 2010, Comput. Hum. Behav..

[45]  R. Rothenberg,et al.  Risk network structure in the early epidemic phase of HIV transmission in Colorado Springs , 2002, Sexually transmitted infections.

[46]  Kenneth Y. Goldberg,et al.  Eigentaste: A Constant Time Collaborative Filtering Algorithm , 2001, Information Retrieval.

[47]  Gert Sabidussi,et al.  The centrality index of a graph , 1966 .

[48]  Andreas Harth,et al.  SIOC: an approach to connect web-based communities , 2006, Int. J. Web Based Communities.

[49]  Alessandro Micarelli,et al.  User Profiles for Personalized Information Access , 2007, The Adaptive Web.

[50]  George Karypis,et al.  Evaluation of Item-Based Top-N Recommendation Algorithms , 2001, CIKM '01.

[51]  Adam Rae,et al.  Improving tag recommendation using social networks , 2010, RIAO.

[52]  Wei-Ying Ma,et al.  AnnoSearch: Image Auto-Annotation by Search , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[53]  A. Hasan The Unplanned Revolution: Observations on the Processes of Socio-Economic Change in Pakistan , 2009 .

[54]  Stefano Mizzaro,et al.  A social approach to context-aware retrieval , 2011, World Wide Web.

[55]  Bill N. Schilit,et al.  Disseminating active map information to mobile hosts , 1994, IEEE Network.

[56]  Pinar Duygulu Sahin,et al.  Tag Suggestr: Automatic Photo Tag Expansion Using Visual Information for Photo Sharing Websites , 2008, SAMT.

[57]  David Carmel,et al.  Social media recommendation based on people and tags , 2010, SIGIR.

[58]  A. Kaplan,et al.  Users of the world, unite! The challenges and opportunities of Social Media , 2010 .

[59]  Mark S. Granovetter The Strength of Weak Ties , 1973, American Journal of Sociology.

[60]  Mor Naaman,et al.  Zonetag: Designing context-aware mobile media capture to increase participation , 2006 .

[61]  Bo Hu,et al.  Ontology-based medical image annotation with description logics , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[62]  Peter J. Denning,et al.  Is computer science science? , 2005, CACM.

[63]  S. Feld Why Your Friends Have More Friends Than You Do , 1991, American Journal of Sociology.

[64]  Christian Bizer,et al.  DBpedia spotlight: shedding light on the web of documents , 2011, I-Semantics '11.

[65]  Rodrigo F. Carvalho,et al.  Metadata goes where Metadata is : contextual networks in the photographic domain , 2008 .

[66]  Bradley N. Miller,et al.  PocketLens: Toward a personal recommender system , 2004, TOIS.

[67]  Amanda Spink,et al.  Issues of context in information retrieval (IR): an introduction to the special issue , 2002, Inf. Process. Manag..

[68]  Gregory D. Abowd,et al.  The context toolkit: aiding the development of context-enabled applications , 1999, CHI '99.

[69]  Albrecht Schmidt,et al.  Advanced Interaction in Context , 1999, HUC.

[70]  Nigel Shadbolt,et al.  Image annotation with Photocopain , 2006 .

[71]  P. Abell Social Networks in Urban Situations: Analyses of Personal Relationships in Central African Towns , 1971 .

[72]  Mor Naaman,et al.  Leveraging context to resolve identity in photo albums , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[73]  Wan-Shiou Yang,et al.  Mining Social Networks for Targeted Advertising , 2006, Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06).

[74]  Ramez Elmasri,et al.  Fundamentals of Database Systems , 1989 .

[75]  Cliff Lampe,et al.  A face(book) in the crowd: social Searching vs. social browsing , 2006, CSCW '06.

[76]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .

[77]  Alan F. Smeaton,et al.  MediAssist: Using Content-Based Analysis and Context to Manage Personal Photo Collections , 2006, CIVR.

[78]  Orri Erling,et al.  RDF Support in the Virtuoso DBMS , 2007, CSSW.

[79]  G. Tomlinson,et al.  YouTube as a source of information on immunization: a content analysis. , 2007, JAMA.

[80]  Arnold W. M. Smeulders,et al.  PicToSeek: combining color and shape invariant features for image retrieval , 2000, IEEE Trans. Image Process..

[81]  Jeremy J. Carroll,et al.  Resource description framework (rdf) concepts and abstract syntax , 2003 .

[82]  B. S. Manjunath,et al.  Not all tags are created equal: Learning flickr tag semantics for global annotation , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[83]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[84]  Paul D. Clough,et al.  Easy on that trigger dad: a study of long term family photo retrieval , 2009, Personal and Ubiquitous Computing.

[85]  D. Boyd Why Youth (Heart) Social Network Sites: The Role of Networked Publics in Teenage Social Life , 2007 .

[86]  Joan C. Nordbotten,et al.  The Role of Context in Image Interpretation , 2007, CIR.

[87]  John G. Breslin,et al.  Tag Mediated Society with SCOT Ontology , 2007, Semantic Web Challenge.

[88]  Bill N. Schilit,et al.  Context-aware computing applications , 1994, Workshop on Mobile Computing Systems and Applications.

[89]  C. Chronaki,et al.  European citizens' use of E-health services: A study of seven countries , 2007, BMC public health.

[90]  Chihli Hung and Chih-Fong Tsai,et al.  Automatically Annotating Images with Keywords: A Review of Image Annotation Systems , 2008 .

[91]  John McCarthy,et al.  Circumscription—a form of non-monotonic reasoning , 1987 .

[92]  Liang-Tien Chia,et al.  Automatic image tagging via category label and web data , 2010, ACM Multimedia.

[93]  J. A. Barnes Class and Committees in a Norwegian Island Parish , 1954 .

[94]  Edward J. Caropreso,et al.  Effects of Personality on Small Group Communication and Task Engagement in an , 2005 .

[95]  Samson W. Tu,et al.  Writing Rules for the Semantic Web Using SWRL and Jess , 2005 .

[96]  Claudio Gutiérrez,et al.  Representing, Querying and Transforming Social Networks with RDF/SPARQL , 2009, ESWC.

[97]  James A. Hendler,et al.  DAML+OIL: An Ontology Language for the Semantic Web , 2002, IEEE Intell. Syst..

[98]  N. Belkin,et al.  Reconsideration of the simulated work task situation , 2010 .

[99]  Luis Fernández-Luque,et al.  An Analysis of Personal Medical Information Disclosed in YouTube Videos Created by Patients with Multiple Sclerosis , 2009, MIE.

[100]  Njål T. Borch,et al.  An Easy to Use and Affordable Home-Based Personal eHealth System for Chronic Disease Management Based on Free Open Source Software , 2008, MIE.

[101]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994, Structural analysis in the social sciences.

[102]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[103]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Electronic Imaging.

[104]  Hans Peter Luhn,et al.  A Statistical Approach to Mechanized Encoding and Searching of Literary Information , 1957, IBM J. Res. Dev..

[105]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[106]  Dennis M. Murphy,et al.  Propaganda: Can a Word Decide a War? , 2007, The US Army War College Quarterly: Parameters.

[107]  Guanling Chen,et al.  A Survey of Context-Aware Mobile Computing Research , 2000 .

[108]  Ramesh C. Jain,et al.  Classification and annotation of digital photos using optical context data , 2008, CIVR '08.

[109]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[110]  P. Ormerod,et al.  Social network markets: a new definition of the creative industries , 2008, Journal of Cultural Economics.

[111]  Manfred Tscheligi,et al.  CHI '04 Extended Abstracts on Human Factors in Computing Systems , 2004, CHI 2004.

[112]  S. Avineri,et al.  The Social and Political Thought of Karl Marx , 1968 .

[113]  Jane Yung-jen Hsu,et al.  Tag-Based User Profiling for Social Media Recommendation , 2008 .

[114]  Tim Berners-Lee,et al.  Linked Data - The Story So Far , 2009, Int. J. Semantic Web Inf. Syst..

[115]  J. Wiest,et al.  The Arab Spring| Social Media in the Egyptian Revolution: Reconsidering Resource Mobilization Theory , 2011 .

[116]  Mor Naaman,et al.  ZoneTag's Collaborative Tag Suggestions: What is This Person Doing in My Phone? , 2008, IEEE MultiMedia.

[117]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[118]  Luciano Rossoni,et al.  Models and methods in social network analysis , 2006 .

[119]  Fernando Ortega,et al.  Improving collaborative filtering recommender system results and performance using genetic algorithms , 2011, Knowl. Based Syst..

[120]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[121]  V. Hougham Sociological Skills Used in the Capture of Saddam Hussein , 2005 .

[122]  S. Ahmad Evidence of the characteristics of women entrepreneurs in the Kingdom of Saudi Arabia , 2011 .

[123]  Wendy E. Mackay,et al.  Triggers and barriers to customizing software , 1991, CHI.

[124]  Eckenhofer Eva Maria Is Centrality the Key to High Performance , 2010 .

[125]  Jason Bonander Personally Tailored Health Information: A Health 2.0 Approach , 2008 .

[126]  Fabien L. Gandon,et al.  State of the Art on Social Network Analysis and its Applications on a Semantic Web , 2008, SDoW@ISWC.

[127]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[128]  Andreas Hotho,et al.  FolkRank : A Ranking Algorithm for Folksonomies , 2006, LWA.

[129]  Simon King,et al.  From context to content: leveraging context to infer media metadata , 2004, MULTIMEDIA '04.

[130]  Detlef Schoder,et al.  Web Science 2.0: Identifying Trends through Semantic Social Network Analysis , 2008, 2009 International Conference on Computational Science and Engineering.

[131]  Grzegorz Kukla,et al.  Recommendation Boosted Query Propagation in the Social Network , 2010, SocInfo.

[132]  Adrian Burton,et al.  YouTube-ing your way to neurological knowledge , 2008, The Lancet Neurology.

[133]  Eric Prud'hommeaux,et al.  Annotea: an open RDF infrastructure for shared Web annotations , 2002, Comput. Networks.

[134]  Noel E. O'Connor,et al.  Automated Annotation of Landmark Images Using Community Contributed Datasets and Web Resources , 2010, SAMT.

[135]  Chuan Qin,et al.  TagSense: a smartphone-based approach to automatic image tagging , 2011, MobiSys '11.

[136]  Bernardo A. Huberman,et al.  Rhythms of social interaction: messaging within a massive online network , 2006, ArXiv.

[137]  Wolfgang Nejdl,et al.  The Benefit of Using Tag-Based Profiles , 2007, 2007 Latin American Web Conference (LA-WEB 2007).

[138]  Vladimir Pavlovic,et al.  Baselines for Image Annotation , 2010, International Journal of Computer Vision.

[139]  Peter Ingwersen,et al.  Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory , 1996, J. Documentation.

[140]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.