A visual language for non-WIMP user interfaces

Unlike current GUI or WIMP style interfaces, non-WIMP user interfaces, such as virtual environments, involve parallel, continuous interactions with the user. However, most current visual (and non-visual) languages for describing human-computer interaction are based on serial, discrete, token-based models. The paper introduces a visual language for describing and programming the fine-grained aspects of non-WIMP interaction. It is based on the notion that the essence of a non-WIMP dialogue is a set of continuous relationships, most of which are temporary. The underlying model combines a data-flow or constraint-like component for the continuous relationships with an event-based component for discrete interactions, which can enable or disable individual continuous relationships. The language thus separates non-WIMP interaction into two components, each based on existing visual language approaches, and then provides a framework for connecting the two.

[1]  Brad A. Myers,et al.  User interface software tools , 1995, TCHI.

[2]  Robert J. K. Jacob,et al.  A State Transition Diagram Language for Visual Programming , 1985, Computer.

[3]  Ralph D. Hill,et al.  Supporting concurrency, communication, and synchronization in human-computer interaction—the Sassafras UIMS , 1986, TOGS.

[4]  Stéphane Chatty,et al.  Extending a graphical toolkit for two-handed interaction , 1994, UIST '94.

[5]  William M. Newman,et al.  A system for interactive graphical programming , 1968, AFIPS Spring Joint Computing Conference.

[6]  Robert J. K. Jacob,et al.  Using formal specifications in the design of a human-computer interface , 1983, CACM.

[7]  Robert J. K. Jacob,et al.  A Specification Language for Direct-Manipulation User Interfaces , 1986, ACM Trans. Graph..

[8]  Gregory D. Abowd,et al.  Integrating status and event phenomena in formal specifications of interactive systems , 1994, SIGSOFT '94.

[9]  Michael Kass,et al.  CONDOR: constraint-based dataflow , 1992, SIGGRAPH.

[10]  Frederick P. Brooks,et al.  VIEW: an exploratory molecular visualization system with user-definable interaction sequences , 1993, SIGGRAPH.

[11]  Ricky Yeung,et al.  TBAG: a high level framework for interactive, animated 3D graphics applications , 1994, SIGGRAPH.

[12]  John F. Hughes,et al.  An architecture for an extensible 3D interface toolkit , 1994, UIST '94.

[13]  Mark Green,et al.  The University of Alberta user interface management system , 1985, SIGGRAPH.

[14]  Jakob Nielsen,et al.  Noncommand user interfaces , 1993, CACM.

[15]  Steven L. Rohall,et al.  The Rendezvous architecture and language for constructing multiuser applications , 1994, TCHI.

[16]  Jock D. Mackinlay,et al.  A Semantic Analysis of the Design Space of Input Devices , 1990, Hum. Comput. Interact..

[17]  John L. Sibert,et al.  An object-oriented user interface management system , 1986, SIGGRAPH.

[18]  Dan R. Olsen,et al.  User Interface Management Systems: Models and Algorithms , 1991 .

[19]  Deborah Hix,et al.  Pre-screen projection: from concept to testing of a new interaction technique , 1995, CHI '95.

[20]  Michael Gleicher,et al.  A graphics toolkit based on differential constraints , 1993, UIST '93.

[21]  James D. Hollan,et al.  Pad++: advances in multiscale interfaces , 1994, CHI Conference Companion.

[22]  Mark Green,et al.  SIGGRAPH '90 Workshop report: software architectures and metaphors for non-WIMP user interfaces , 1991, COMG.

[23]  David A. Carr,et al.  Specification of interface interaction objects , 1994, CHI '94.

[24]  Wayne Citrin,et al.  The Design of a Completely Visual Object-Oriented Programming Language , 1994 .

[25]  Daniel T. Ling,et al.  Dialogue structures for virtual worlds , 1991, CHI.

[26]  David A. Carr,et al.  Using Interaction Object Graphs to Specify and Develop Graphical Widgets , 1994 .