SpaceCase: A Model of Spatial Preposition Use

SpaceCase: A Model of Spatial Preposition Use Kate Lockwood (kate@cs.northwestern.edu), Ken Forbus (forbus@northwestern.edu) and Jeffrey Usher (usher@northwestern.edu) Department of Computer Science, 1890 Maple Ave. Evanston, IL 60201 USA (2003) labeling experiment, including a sensitivity analysis that indicates the model is working for the right reasons. We show how SpaceCase models Feist and Gentner’s (2001) retrieval results next. Finally, we discuss related and future work. Abstract We present SpaceCase, a computational model of spatial preposition use that combines geometric and functional influences. SpaceCase treats spatial preposition use as governed by evidential rules, each representing influences of particular factors. Our model is unique in relying on both automatically constructed visual representations from sketched input, and drawing our functional representations from an independently derived large knowledge base, both of which reduce tailorability. SpaceCase can account for the results of Feist and Gentner (2003), whose experiments about in/on judgments in native English speakers showed the influence of four factors: (1) geometry of the ground, (2) animacy of the ground (3) animacy of the figure and (4) function of the ground. SpaceCase also captures Feist and Gentner’s (2001) result that memory for spatial relationships can be influenced by spatial language during encoding. Psychological Evidence Keywords: Spatial language, Bayesian models Introduction Our aptitude for communicating and reasoning about space is key to our abilities to navigate, give directions , and to reason analogically about other subjects (Gentner, Imai, & Boroditsky, 2002). One way that we describe spatial scenes is through the use of prepositions like in and on. Traditionally, it was thought that the spatial preposition used to describe a scene depended solely on the geometric arrangement and properties of the objects in the scene. As described below, however, recent research indicates that non-geometric properties also play important roles. This raises a new difficulty for modeling the use of spatial prepositions: Representations of these other factors need to be created, ideally created independently from the spatial preposition model itself, to reduce tailorability. The same is true of course for the spatial representations used in stimuli given to models. Fortunately, progress in Artificial Intelligence has provided off-the-shelf knowledge bases and sketching systems with reasonable stand-ins for visual processing abilities. SpaceCase exploits both, by contrast with all previous models that we are aware of. We begin by reviewing some of the evidence about spatial prepositions, focusing on the Feist and Gentner (2001, 2003) experiments. Next we describe SpaceCase, showing how it uses an independently-motivated sketch understanding system (sKEA, (Forbus & Usher, 2002) ) and draws its representations from a large knowledge base (over 39,000 concepts, constrained by 1.2 million facts). We next show how SpaceCase can account for the Feist and Gentner The issue of how language and space interact has had a long history in cognitive science research. Early theories of spatial preposition use claimed that people assigned spatial prepositions based on the geometry of a visual scene. However, more recent work has shown that the use of spatial prepositions is influenced by a variety of functional factors in addition to the geometry of the situation. Factors such as context (Coventry, 1999; Herskovitz, 1986), functional relationships between the objects (Carlson- Radvansky et al., 1999; Coventry et al., 1994; Vandeloise, 1994), and control relationships (Feist & Gentner, 2003; Garrod et al., 1999) also influence how we use prepositions in everyday language (see Coventry & Garrod, 2004 for a review). We focus here on modeling the results of Feist and Gentner (2003), for concreteness. They examined the role of four factors in in/on determinations in visual scenes involving two objects, a figure (located object) and a ground (reference object): (1) the geometry of the ground, (2) the animacy of the ground, (3) the animacy of the figure, and (4) the functional role of the ground. They found that all of these factors were involved in determining whether subjects would describe the figure as on or in the ground. Specifically, high curvature is more likely to lead to in, and low curvature more likely to be associated with on. If the ground were animate (a hand, for instance), in was more likely to be used, whereas if the figure is animate, on was more likely to be used. Moreover, subjects were more likely to use in than on if they were told that the ground was a container (say, a bowl) than when they were told it was something else (e.g., a plate), even with the same curvature. Can language affect how spatial relations are processed? Feist and Gentner (2001) showed that giving subjects a sentence involving a spatial preposition while viewing a scene affects how that scene is stored in memory. That is, suppose a subject is shown Figure 1(right) below while being told “The puppet is on the table.”, as part of a larger set of stimuli. When later asked if they had seen Figure 1(left), which was not shown to them earlier, subjects who had heard on during encoding were more likely to incorrectly report that they had seen it. This suggests that information from multiple modalities (visual and linguistic)