Visual Reasoning in Modeling and Design

Visual Reasoning in Modeling and Design Ashok K. Goel (goel@cc.gatech.edu) School of Interactive Computing, Georgia Institute of Technology 85 Fifth Street, Atlanta, GA 30332, USA Peter C-H. Cheng (P.C.H.Cheng@sussex.ac.uk) Department of Informatics, University of Sussex Falmer, Brighton, BN1 9QH, UK John J. Clement (clement@educ.umass.edu) School of Education and Scientific Reasoning Research Institute, University of Massachusetts Amherst, MA 01003, USA Fehmi Dogan (fehmidogan@iyte.edu.tr) Graduate School of Engineering & Sciences, Izmir Institute of Technology Urla, TR-35430, Izmir, Turkey Mary Hegarty (hegarty@psych.ucsb.edu) Department of Psychology, University of California at Santa Barbara Santa Barbara, California 93106, USA Nancy J. Nersessian (nancyn@cc.gatech.edu) School of Interactive Computing, Georgia Institute of Technology 85 Fifth Street, Atlanta, GA 30332, USA Keywords: Visual reasoning, visual representations, diagrams, mental models, modeling, design. Introduction Visual reasoning plays an important role in modeling in science and engineering, including science and engineering education. It also plays a critical role in engineering and architectural design. Ferguson (1992), for example, views good engineering largely as an outcome of visual reasoning (and not verbal or mathematical reasoning). Yet, there is much that we do not yet understand about visual reasoning. In fact, at present there is not even a universal agreement in cognitive science on a definition of visual representation or reasoning. The purpose of this symposium is to bring together recent cognitive science research on visual reasoning in modeling and design. Our proposal focuses on visual representations such as sketches, drawings and diagrams. The speakers in the proposed symposium represent multiple disciplines within cognitive science, including artificial intelligence (Cheng, Goel), science education (Clement), cognitive psychology (Hegarty), architecture (Dogan), and philosophy and history of science (Nersessian). Goel will be the moderator for the symposium. Presentations Ashok K. Goel Understanding Drawings by Model Construction by Compositional Analogy Understanding sketches, drawings and diagrams is a fundamental problem in visual reasoning. I describe a computational technique for understanding engineering drawings by constructing a teleological model of the target drawing by analogy to the model of a known drawing. Knowledge of the source case is organized in a multimodal schema that contains the source drawing and its teleological model represented at multiple levels of abstraction: the lines and intersections in the drawing, the shapes, the structural components and connections, the causal interactions and processes, and the function of the system depicted in the drawing. Given a target drawing and a relevant source case, the technique of compositional analogy first constructs a representation of the lines and the intersections in the target drawing, then uses the mappings at the level of line intersections to transfer the shape representations from the source case to the target, next uses the mappings at the level of shapes to transfer the full teleological model of the depicted system from the source to the target. The Archytas computer system implements this multimodal knowledge representation and the technique for understanding drawings