Collecting Spatial Information for Locations in a Text-to-Scene Conversion System

We investigate using Amazon Mechanical Turk (AMT) for building a low-level description corpus and populating VigNet, a comprehensive semantic resource that we will use in a text-to-scene generation system. To depict a picture of a location, VigNet should contain the knowledge about the typical objects in that location and the arrangements of those objects. Such information is mostly common-sense knowledge that is taken for granted by human beings and is not stated in existing lexical resources and in text corpora. In this paper we focus on collecting objects of locations using AMT. Our results show that it is a promising approach.

[1]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[2]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[3]  Catherine Havasi,et al.  ConceptNet 3 : a Flexible , Multilingual Semantic Network for Common Sense Knowledge , 2007 .

[4]  Chris Callison-Burch,et al.  Creating Speech and Language Data With Amazon’s Mechanical Turk , 2010, Mturk@HLT-NAACL.

[5]  Bob Coyne,et al.  Collecting Semantic Information for Locations in the Scenario-Based Lexical Knowledge Resource of a Text-to-Scene Conversion System , 2011, KES.

[6]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[7]  Bob Coyne,et al.  VigNet: Grounding Language in Graphics using Frame Semantics , 2011, RELMS@ACL.

[8]  Erik T. Mueller,et al.  Open Mind Common Sense: Knowledge Acquisition from the General Public , 2002, OTM.

[9]  Julia Hirschberg,et al.  Frame Semantics in Text-to-Scene Generation , 2010, KES.

[10]  Richard Sproat,et al.  WordsEye: an automatic text-to-scene conversion system , 2001, SIGGRAPH.

[11]  Peter D. Turney.  Expressing Implicit Semantic Relations without Supervision , 2006, ACL.

[12]  Michael L. Littman,et al.  Corpus-based Learning of Analogies and Semantic Relations , 2005, Machine Learning.

[13]  Richard Sproat.  Inferring the environment in a text-to-scene conversion system , 2001, K-CAP '01.

[14]  Alla Rozovskaya,et al.  UIUC: A Knowledge-rich Approach to Identifying Semantic Relations between Nominals , 2007, ACL 2007.