Automated Evaluation of Annotators for Museum Collections Using Subjective Logic

Museums are rapidly digitizing their collections, and face a huge challenge to annotate every digitized artifact in store. Therefore they are opening up their archives for receiving annotations from experts world-wide. This paper presents an architecture for choosing the most eligible set of annotators for a given artifact, based on semantic relatedness measures between the subject matter of the artifact and topics of expertise of the annotators. We also employ mechanisms for evaluating the quality of provided annotations, and constantly manage and update the trust, reputation and expertise information of registered annotators.

[1]  Gianluca Demartini,et al.  Finding Experts Using Wikipedia , 2007, FEWS.

[2]  Junjie Yao,et al.  Routing Questions to the Right Users in Online Communities , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[3]  Wan Fokkink,et al.  A Trust Model to Estimate the Quality of Annotations using the Web , 2010 .

[4]  Lora Aroyo,et al.  Using Semantic Relations for Content-based Recommender Systems in Cultural Heritage , 2009, WOP.

[5]  John G. Breslin,et al.  Finding experts using Internet-based discussions in online communities and associated social networks , 2007 .

[6]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[7]  Jeannine Mjoseth,et al.  Institute of Museum and Library Services (IMLS) , 2010 .

[8]  Lora Aroyo,et al.  Semantic relations for content-based recommendations , 2009, K-CAP '09.

[9]  Audun Jøsang,et al.  A Logic for Uncertain Probabilities , 2001, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[10]  Motta.  The Hoonoh Ontology for describing Trust Relationships in Information Seeking , 2008 .

[11]  Lora Aroyo,et al.  Semantics : Science , Services and Agents on the World Wide Web , 2008 .

[12]  Susan C. Herring,et al.  Collaborative Authoring on the Web: A Genre Analysis of Online Encyclopedias , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[13]  Antoine Isaac,et al.  SKOS Simple Knowledge Organization System Primer , 2009 .

[14]  John Riedl,et al.  SuggestBot: using intelligent task routing to help people find work in wikipedia , 2007, IUI '07.

[15]  Bob J. Wielinga,et al.  Patterns of semantic relations to improve image content search , 2007, J. Web Semant..

[16]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.