Graflog: a theory of semantics for graphics with applications to human-computer interaction and cad systems

In this dissertation a theory of semantics for graphics for Human-Computer Interaction and CAD systems is presented. This theory allows the integration of interactive computer graphics and natural language processing facilities. Through this theory, the 'knowledge' of both natural language and graphical expressions is represented in an integrated fashion. The semantics of the representational language is explained as an algebraic system that has a dual algorithmic interpretation. In the algebraic domain, the meaning of drawings is explained in terms of a modal multi-sorted algebraic system. The algorithmic interpretation, on the other hand, is used for integrating the declarative semantics of the representational language with the algorithmic knowledge that is required for producing synthetic pictures in computer graphics. This theory was developed in conjunction with a computer program called GRAFLOG. The program is an interactive drafting system that has been augmented with a natural language processing facility. The interactive concepts that emerge in this kind of interface as well as the architecture of the program are illustrated and discussed. The grammatical formalism for handling the natural language facility, and the computational linguistic tools needed for its implementation are also shown. The theory and the program are applied to a simple design task: geometric reasoning for geometric modelling in the wire-frame drawing domain. For this purpose a representational language for expressing design knowledge in this domain is developed. The language is useful for defining the notions of design concept, design intention, and design intention satisfaction. The language is used not only for following up the consequences of the information that is expressed through graphics and natural language, but also for interpreting design intentions that are expressed by human-users, solving in this way design problems that occur in the course of an interactive design session. The representational system is used as well for producing a graphics and natural language explanation of the methods by which the system comes to the solution of design problems. In the last chapter, a reflection on the relation of computer graphics and AI is presented.