Modeling visual interactive systems through dynamic visual languages

Visual interaction processes are modeled in this paper as sequences of visual sentences in which for each visual sentence only a limited set of user actions is possible. We introduce the notion of "dynamic visual language" as a weakly ordered set of visual sentences characterized by the presence of common elements. We present a formal model of derivation of visual sentences in a dynamic visual language in which each visual sentence specifies the possible actions which can be performed on it and the possible transformations it can go through. In this way, we offer a formal setting in which the interaction process can be formally specified. A user interface can be derived from the formal specification, so that it embeds proper context elements which limit user disorientation. The concepts are illustrated by the user interaction with a prototype of a digital library developed at the University of Bari.

[1]  Alessandro Bianchi,et al.  Designing usable visual languages: the case of immune system studies , 1999, Proceedings 1999 IEEE Symposium on Visual Languages.

[2]  Shi-Kuo Chang,et al.  A framework of a visual language with dynamic specification , 1995, Proceedings of Symposium on Visual Languages.

[3]  Bernd Meyer,et al.  A constraint-based framework for diagrammatic reasoning , 2000, Appl. Artif. Intell..

[4]  Gabriele Taentzer,et al.  Hierarchically Distributed Graph Transformation , 1994, TAGT.

[5]  Piero Mussio,et al.  Defining visual languages for interactive computing , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[6]  Shi-Kuo Chang.  Dynamic visual languages , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[7]  Maria Francesca Costabile,et al.  Towards formal measures of usability for visual interactive systems , 2002, Proceedings IEEE 2002 Symposia on Human Centric Computing Languages and Environments.

[8]  Alexander Repenning,et al.  Agentsheets: A Medium for Creating Domain-Oriented Languages , 1995, Computer.

[9]  Philip T. Cox,et al.  A declarative language for the design of structures , 1997, Proceedings. 1997 IEEE Symposium on Visual Languages (Cat. No.97TB100180).

[10]  Francesco Parisi-Presicce,et al.  Visual Multiset Rewriting: Applications to Diagram Parsing and Reasoning , 2000, WMP.

[11]  Ladislav Tondl,et al.  Problems of semantics , 1981 .

[12]  Kim Marriott,et al.  Specifying diagram animation with rewrite systems , 1997 .

[13]  Wieslaw Zielonka,et al.  Notes on Finite Asynchronous Automata , 1987, RAIRO Theor. Informatics Appl..

[14]  Genny Tortora,et al.  A Parsing Methodology for the Implementation of Visual Systems , 1997, IEEE Trans. Software Eng..

[15]  Giancarlo Mauri,et al.  Coordination through Group Rewriting , 1999, Grammatical Models of Multi-Agent Systems.

[16]  Mark Minas.  Automatically generating environments for dynamic diagram languages , 1998, Proceedings. 1998 IEEE Symposium on Visual Languages (Cat. No.98TB100254).

[17]  Gabriele Taentzer,et al.  The AGG approach: language and environment , 1999 .

[18]  Piero Mussio,et al.  Cooperative Visual Environments for the Design of Effective Visual Systems , 1993, J. Vis. Lang. Comput..

[19]  Albert Zündorf,et al.  The PROGRES approach: language and environment , 1999 .

[20]  Piero Mussio,et al.  On Orders in Visual Interaction , 1999, J. Vis. Lang. Comput..

[21]  Umberto Cugini,et al.  Issues in the combination of visual and haptic interaction , 2001, HCI.

[22]  James C. Spohrer,et al.  KidSim: programming agents without a programming language , 1994, CACM.

[23]  Alexander Repenning,et al.  Behaviour processors: layers between end-users and Java virtual machines , 1997, Proceedings. 1997 IEEE Symposium on Visual Languages (Cat. No.97TB100180).

[24]  Piero Mussio,et al.  Specification and dialogue control of visual interaction through visual rewriting systems , 1999, TOPL.

[25]  Andy Schürr,et al.  A graph based framework for the implementation of visual environments , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[26]  Eric J. Golin,et al.  Parsing visual languages with picture layout grammars , 1991, J. Vis. Lang. Comput..

[27]  Alan J. Dix,et al.  Towards a Ubiquitous Semantics of Interaction: Phenomenology, Scenarios, and Traces , 2002, DSV-IS.

[28]  Daniela Fogli,et al.  Toward overcoming culture, skill and situation hurdles in Human-Computer Interaction , 2002, Universal Access in the Information Society.

[29]  Nicola Fanizzi,et al.  An adaptive visual environment for digital libraries , 1999, International Journal on Digital Libraries.