A Web-Based Infrastructure for the Assisted Annotation of Heritage Collections

Annotations provide a valuable perspective on the semantic information present in digital heritage collections, and in recent years they've been employed in a number of innovative, user-centric techniques that can personalise a user's experience of heritage materials, such as by actively adapting exhibits as a user reveals their interests, or by guiding users to explore collections which are meaningfully linked to what they have previously encountered. Despite the captivating opportunities offered by these techniques, collecting annotations for a large heritage collection is no trivial task. A significant amount of work is required to manually annotate large quantities of heritage materials, and automated, computational approaches leave much to be desired regarding the level of insight and semantic richness that they can currently provide. By analysing the emergent relationships between the initial annotations in a collection, we propose a metadata-driven algorithm for assisting and augmenting the annotation process. This algorithm, called SAGA (Semantically-Annotated Graph Analysis), allows for semi-automatic annotation, which balances the value of the contributions of human annotators with the time and effort-saving benefits of an automatic, suggestion-driven process. SAGA uses an entity relationship-driven approach to make annotation suggestions. It is used in the context of a web-based infrastructure called SAGE (Semantic Annotation by Group Exploration), a multiagent environment which assists groups of experts in creating comprehensive annotation sets for heritage collections. SAGA and SAGE are evaluated from the perspectives of suggestion accuracy, explicit user acceptance and implicit user acceptance, and demonstrate strong results in each evaluation.

[1]  Antoine Isaac,et al.  Europeana Linked Open Data - data.europeana.eu , 2013, Semantic Web.

[2]  James Bennett,et al.  The Netflix Prize , 2007 .

[3]  Nicola Orio,et al.  The CULTURA Project: Supporting Next Generation Interaction with Digital Cultural Heritage Collections , 2012, EuroMed.

[4]  Wolfgang Nejdl,et al.  Bringing order to your photos: event-driven classification of flickr images based on social knowledge , 2010, CIKM.

[5]  Dong Liu,et al.  Tag ranking , 2009, WWW '09.

[6]  H. Van Dyke Parunak,et al.  Changing Roles: Dynamic Role Assignment , 2003, J. Object Technol..

[7]  Danail Bonchev,et al.  Quantitative Measures of Network Complexity , 2005 .

[8]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[9]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[10]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[11]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[12]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[13]  Russell Beale,et al.  Supporting serendipity: Using ambient intelligence to augment user exploration for data mining and web browsing , 2007, Int. J. Hum. Comput. Stud..

[14]  Elaine Toms,et al.  Cultural Heritage in CLEF (CHiC) 2013 , 2013, CLEF.

[15]  Sourav S. Bhowmick,et al.  Content is still king: the effect of neighbor voting schemes on tag relevance for social image retrieval , 2012, ICMR.

[16]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[17]  Bingbing Ni,et al.  Assistive tagging: A survey of multimedia tagging with human-computer joint exploration , 2012, CSUR.

[18]  Dafna Shahaf,et al.  Towards a Theory of AI Completeness , 2007, AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning.

[19]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[20]  Séamus Lawless,et al.  ENRICH 2013 , 2013, SIGIR Forum.

[21]  Kristen Grauman,et al.  What's it going to cost you?: Predicting effort vs. informativeness for multi-label image annotations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Jonathon S. Hare,et al.  Mind the gap: another look at the problem of the semantic gap in image retrieval , 2006, Electronic Imaging.

[23]  Gerhard Fischer,et al.  Cultures of Participation and Social Computing: Rethinking and Reinventing Learning and Education , 2009, 2009 Ninth IEEE International Conference on Advanced Learning Technologies.

[24]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .

[25]  Adam Rae,et al.  Improving tag recommendation using social networks , 2010, RIAO.

[26]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[27]  Harri Oinas-Kukkonen,et al.  Social Networks and Information Systems: Ongoing and Future Research Streams , 2010, J. Assoc. Inf. Syst..

[28]  Benno Stein,et al.  Information Access Evaluation. Multilinguality, Multimodality, and Visualization , 2013, Lecture Notes in Computer Science.

[29]  Jonathon S. Hare,et al.  Bridging the Semantic Gap in Multimedia Information Retrieval: Top-down and Bottom-up approaches , 2006 .

[30]  Martin Halvey,et al.  Is relevance hard work?: evaluating the effort of making relevant assessments , 2013, SIGIR.

[31]  Michael Wooldridge,et al.  Handbook of Knowledge Representation Edited Situation Calculus , 2022 .

[32]  Eneko Agirre,et al.  PATHS: A System for Accessing Cultural Heritage Collections , 2013, ACL.

[33]  Nicola Ferro,et al.  A formal model of annotations of digital content , 2007, TOIS.

[34]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[35]  Rui Li,et al.  Survey on social tagging techniques , 2010, SKDD.

[36]  Andrea Marchetti,et al.  SemKey: A Semantic Collaborative Tagging System , 2007 .

[37]  Steffen Staab,et al.  Semantic Annotation of Images and Videos for Multimedia Analysis , 2005, ESWC.

[38]  Nicola Ferro,et al.  Cultural Heritage in CLEF (CHiC) , 2013 .

[39]  William I. Grosky,et al.  Idea Grou p Inc . Copy right Idea Grou p Inc . Copy right Idea Grou p Inc . Copy right Idea Grou p Inc . Chapter II Bridging the Semantic Gap in Image Retrieval , 2018 .

[40]  Tsvi Kuflik,et al.  Personalization in cultural heritage: the road travelled and the one ahead , 2011, User Modeling and User-Adapted Interaction.

[41]  Nenghai Yu,et al.  Learning to tag , 2009, WWW '09.

[42]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[43]  Mark Stevenson,et al.  Adapting Wikification to Cultural Heritage , 2012, LaTeCH@EACL.

[44]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.

[45]  Mark Stevenson,et al.  Computing similarity between items in a digital library of cultural heritage , 2013, JOCCH.

[46]  Sourav S. Bhowmick,et al.  Social image tag recommendation by concept matching , 2011, ACM Multimedia.

[47]  Marcel Worring,et al.  Learning Social Tag Relevance by Neighbor Voting , 2009, IEEE Transactions on Multimedia.