Visual Analogies at Multiple Levels of Abstraction

Visual Analogies at Multiple Levels of Abstraction Patrick W. Yaner (yaner@cc.gatech.edu) Ashok K. Goel (goel@cc.gatech.edu) Design Intelligence Laboratory School of Interactive Computing, Georgia Institute of Technology Atlanta, GA 30332-0760 Abstract We describe a method for constructing a teleological model of an unlabelled 2D line drawing by analogy to a known model of a drawing with similar structure. The source case is repre- sented as a schema that contains its line drawing and its teleo- logical model represented at multiple levels of abstraction: the lines and intersections in the drawing, the shapes, the structural components and connections, the causal interactions and pro- cesses, and the function of the device depicted in the drawing. Given a target drawing and a relevant source case, our method of compositional analogy first constructs a representation of the lines and the intersections in the target drawing, then uses the mappings at the level of line intersections to transfer the shape representations from the source case to the target, next uses the mappings at the level of shapes to transfer the full tele- ological model of the depicted system from the source to the target. Keywords: diagrammatic reasoning; analogy; visual reason- ing; case-based reasoning; design a b c Motivation and Goals Humans often communicate with drawings, and human de- signers are able to read and understand drawings of designs. When shown a drawing of a new device, an expert designer in the device domain may not only identify device compo- nents but also explain what the device does and how it works. We have developed a theory of how this might be done: by transfer of a teleological model of a known drawing with a similar or related structure. This theory is implemented in a program called Archytas. This program reads in a 2D un- labelled drawing and, given the source drawing and associ- ated teleological model, attempts to infer by analogy, (1) a representation of the shapes and spatial relations in the tar- get drawing, (2) a representation of the structural components and connections of the device depicted in the drawing, (3) a qualitative representation of the causal interactions and pro- cesses in the device, and (4) a representation of the function of the device. This method of compositional analogy works iteratively to successively higher levels of abstraction, inter- leaving mapping and transfer at various levels to construct a new teleological model. To see the need for mapping and transfer at multiple ab- straction levels, let us consider the task of mapping the source drawing illustrated in figure 1(a) to the similar target drawing illustrated in figure 1(b). If we treat the problem as one of first recognizing the geometric elements and spatial relations among them, then we can treat this representation as a la- belled graph: A contains B, C is adjacent to D, and so on. A graph-theoretic method for analogy-based recognition may Figure 1: (a) A sample source drawing of a piston and crankshaft assembly. (b) A target drawing of the same de- vice with the piston now at the top of its range of motion, and (c) another sample target drawing of a two piston and crankshaft assembly. The shape patterns as well as the SBF model is transferred from the source (a) to an input drawing such as (b) or (c). then be used to find a consistent mapping between the graphs representing the source and target drawings. However, the method runs into difficulty for the target drawing shown in figure 1(c) with figure 1(a) as the source drawing. In this problem, the number of components, and thus the number of shapes, is different, and either the graph-theoretic method would have to relax the constraint of one-to-one mapping, or else the analogy would have to be performed twice in order to transfer a model successfully from figure 1(a) to figure 1(c). Figure 2 illustrates a similar example from the domain of door latches. To address the above difficulties, our method of compo- sitional analogy performs analogy at multiple levels of ab- straction. The analogical mapping and transfer is enabled by organizing knowledge of the source case at multiple ab- straction levels. Figure 3 illustrates the knowledge organi- zation in a source case. The function, behavior, structure, shape, and drawing in a source case form an abstraction hi-