Defining visual languages for interactive computing

A novel definition of visual languages allows a uniform approach to satisfying the needs of visual reasoning faced in visual human-computer interaction. The way the machine associates a computational meaning with an image, and conversely, the way it generates an image on the screen from a computation are formally described. A definition of a visual sentence and of a visual language as a set of visual sentences is discussed. A hierarchy of visual languages is derived in relation to the requirements for intelligible, manageable and trustable interaction between humans and computers.

[1]  P. Mussio,et al.  The design of Anthropocentric Cooperative Visual Environments , 1995, Proceedings of Symposium on Visual Languages.

[2]  Bing Yu,et al.  The SIL-Icon Compiler - an Icon-Oriented System Generator , 1988, Int. J. Pattern Recognit. Artif. Intell..

[3]  Margaret M. Burnett,et al.  Interactive Visual Data Abstraction in a Declarative Visual Programming Language , 1994, J. Vis. Lang. Comput..

[4]  Eser Kandogan,et al.  Elastic windows: improved spatial layout and rapid multiple window operations , 1996, AVI '96.

[5]  Grzegorz Rozenberg,et al.  Using String Languages to Describe Picture Languages , 1982, Inf. Control..

[6]  Piero Mussio,et al.  Cooperative Visual Environments for the Design of Effective Visual Systems , 1993, J. Vis. Lang. Comput..

[7]  King-Sun Fu,et al.  Attributed Grammar-A Tool for Combining Syntactic and Statistical Approaches to Pattern Recognition , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  H. W. Zeevat,et al.  Reasoning with Diagrammatic Representations , 1995 .

[9]  Bing Yu,et al.  Icon Purity - toward a Formal Theory of Icons , 1987, Int. J. Pattern Recognit. Artif. Intell..

[10]  N. Hari Narayanan,et al.  Diagrammatic Reasoning: Cognitive and Computational Perspectives , 1995 .

[11]  Piero Mussio,et al.  Visual conditional attributed rewriting systems in visual language specification , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[12]  S. Levialdi,et al.  A visual approach to HCI , 1996, SGCH.

[13]  Christine M. Mitchell,et al.  Human-computer cooperative problem solving: theory, design, and evaluation of an intelligent associate system , 1995, IEEE Trans. Syst. Man Cybern..

[14]  Piero Mussio,et al.  Simulation of hepatological models: a study in visual interactive exploration of scientific problems , 1991, J. Vis. Lang. Comput..

[15]  Kim Marriott,et al.  A declarative specification and semantics for visual languages , 1991, J. Vis. Lang. Comput..

[16]  Katsushi Inoue,et al.  A survey of two-dimensional automata theory , 1991, Inf. Sci..

[17]  Peter Wegner,et al.  Interaction as a basis for empirical computer science , 1995, CSUR.

[18]  Kenneth M. Kahn,et al.  Complete visualizations of concurrent programs and their executions , 1990, Proceedings of the 1990 IEEE Workshop on Visual Languages.

[19]  Antonio Restivo,et al.  Recognizable Picture Languages , 1992, Int. J. Pattern Recognit. Artif. Intell..