Reasoning Visually about Spatial Interactions

This paper is concerned with how diagrams can be used for reasoning about spatial interactions of objects. We describe a computational approach that emulates the human capability of predicting interactions of simple objects depicted in two dimensional diagrams. Three core aspects of this approach are a visual representation scheme that has symbolic and imaginal parts, the use of visual processes to manipulate the imaglnat part and to extract spatial information, and visual cases that encode experiential knowledge and play a central role in the generation of spatial inferences. These aspects are described and the approach is illustrated with an example. Then we show that reasoning with images is an emerging and promising area of investigation by discussing computational and cognitive research on imagery. 1 Introduction Humans quite often make use of spatial information implicit in diagrams to make inferences. For example, anyone famlliar with the operation of gears will be able to solve the problem posed in Figure 1 by imagining the rotary motion of gear1 being transmitted to the rod through gear2, resulting in the horizontal translation of the rod until it hits the wall. In such situations humans reason about spatial interactions not only by using conceptual knowledge, but also by extracting constraints on such interactions from a perceived image. This integrated use of visual knowledge (about spatial configurations) from the diagram and conceptual knowledge (such as the rigidity or plasticity of objects involved) is a very interesting phenomenon, in this paper we illustrate a computational approach that emulates this capability for solving simple motion prediction problems. The class of problems we address is the following: given a two dimensional diagram of the spatial configuration of a set of objects, one or more Initial motions of objects and relevant conceptual information about them, predict the subsequent dynamics of the configuration. Figure 2 shows a typical example.