A compact graphical representation of user interface interaction objects

The design of new user-interface interaction objects (or widgets) remains a laborious process. The designer must translate the proposed widget into a computer language and install it in the graphical user interface. This dissertation proposes an executable, graphical specification method to help solve this problem. After a review of previous methods used to specify user interfaces and widgets, the Interaction Object Graph (IOG) is introduced and defined. IOGs are a graphical specification method designed specifically for widgets. Example IOG specifications are given for many current widgets and for several new widgets. In addition, it is shown that an arbitrary Turing machine can be transformed into an IOG. Therefore, IOGs may be used to specify any widget. The dissertation also reports on a pilot experiment comparing IOGs with a text specification method, User Action Notation. This experiment gives a weak indication that graphical specifications such as IOGs are easier to understand. Finally, a C++ class library was implemented for executing IOG-specified widgets while animating the diagrams. The design of this library and the animated specification diagrams are discussed. IOGs extend statecharts with special states to represent display changes and with special nodes and arcs to model widget attribute updates. Display states are represented by a picture of the widget appearance. The picture improves IOG diagram readability by providing an idea of how the widget's appearance will change as the user operates the widget. Data nodes and arcs allow the widget designer to specify the relationship between user actions and widget attribute values. This capability can result in smaller specifications than methods without data modeling. It is also easier to locate all updates to a particular data item in an IOG than in methods that model change with equations annotating a state or column entry. Finally, in order to enhance specification understanding and debugging, the IOG class library animates an IOG diagram showing changes in the active states as the user operates the widget.

[1]  Andrew F. Monk,et al.  Action-effect rules: a technique for evaluating an informal specification against principles , 1990 .

[2]  Pierre D. Wellner,et al.  Statemaster: A UIMS based on statecharts for prototyping and target implementation , 1989, CHI '89.

[3]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[4]  Ephraim P. Glinert,et al.  Pict: An Interactive Graphical Programming Environment , 1984, Computer.

[5]  Frances Paulisch,et al.  Using constraints to achieve stability in automatic graph layout algorithms , 1990, CHI '90.

[6]  J. I. Helfman.  Panther: a specification system for graphical controls , 1986, CHI '87.

[7]  Matthias Grochtmann,et al.  WINDOWNET - A FORMAL NOTATION FOR WINDOW-BASED USER INTERFACES , 1987 .

[8]  Ellis Horowitz,et al.  A system for specifying and rapidly prototyping user interfaces , 1991 .

[9]  Kevin A. Mayo,et al.  A Framework for Precise, Reusable Task Abstractions , 1994, DSV-IS.

[10]  Ralph D. Hill,et al.  Supporting concurrency, communication, and synchronization in human-computer interaction—the Sassafras UIMS , 1986, TOGS.

[11]  Dan R. Olsen,et al.  Creating interactive techniques by symbolically solving geometric constraints , 1990, UIST '90.

[12]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[13]  Ben Shneiderman,et al.  Tree-maps: a space-filling approach to the visualization of hierarchical information structures , 1991, Proceeding Visualization '91.

[14]  Sougata Mukherjea,et al.  Toward visual debugging: integrating algorithm animation capabilities within a source-level debugger , 1994, TCHI.

[15]  Catherine Plaisant,et al.  Touchscreen toggle design , 1992, CHI.

[16]  Ben Shneiderman,et al.  The alphaslider: a compact and rapid selector , 1994, CHI Conference Companion.

[17]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[18]  Mícheál Mac an Airchinnigh,et al.  Seeheim Workshop on User Interface Management Systems Report of the Working Group on the User's Conceptual Model * , 1985, Comput. Graph. Forum.

[19]  B. Shneiderman,et al.  The dynamic HomeFinder: evaluating dynamic queries in a real-estate information exploration system , 1992, SIGIR '92.

[20]  Dan R. Olsen,et al.  Propositional production systems for dialog description , 1990, CHI '90.

[21]  Gurminder Singh,et al.  A high-level user interface management system , 1989, CHI '89.

[22]  Brad A. Myers,et al.  Creating user interfaces by demonstration , 1988 .

[23]  H. Rex Hartson,et al.  Temporal Aspects of Tasks in the User Action Notation , 1992, Hum. Comput. Interact..

[24]  Gurminder Singh,et al.  Druid: a system for demonstrational rapid user interface development , 1990, UIST '90.

[25]  Robert J.K. Jacob.  Executable specifications for a human-computer interface , 1983, CHI '83.

[26]  Robert J. K. Jacob,et al.  A Specification Language for Direct-Manipulation User Interfaces , 1986, ACM Trans. Graph..

[27]  J.T. Stasko,et al.  Tango: a framework and system for algorithm animation , 1990, Computer.

[28]  Nasa,et al.  A metamodel for the runtime architecture of an interactive system: the UIMS tool developers workshop , 1992, SGCH.

[29]  David Harel,et al.  On visual formalisms , 1988, CACM.

[30]  Amir Pnueli,et al.  On the Formal Semantics of Statecharts (Extended Abstract) , 1987, LICS.

[31]  Q. Mao,et al.  Defining the presentation of application data by a graphical language , 1989, UIST '89.

[32]  Brad A. Myers.  INCENSE: A system for displaying data structures , 1983, SIGGRAPH.

[33]  Bradley T. Vander Zanden.  Constraint grammars–a new model for specifying graphical applications , 1989, CHI '89.

[34]  Dan R. Olsen,et al.  User Interface Management Systems: Models and Algorithms , 1991 .

[35]  Anthony I. Wasserman,et al.  Extending State Transition Diagrams for the Specification of Human–Computer Interaction , 1985, IEEE Transactions on Software Engineering.

[36]  Scott E. Hudson,et al.  Interactive specification of flexible user interface displays , 1990, TOIS.

[37]  Joanne M. Atlee,et al.  State-Based Model Checking of Event-Driven System Requirements , 1993, IEEE Trans. Software Eng..

[38]  A. Borning,et al.  Defining constraints graphically , 1986, CHI '86.

[39]  S. E. Hudson,et al.  Graphical specification of flexible user interface displays , 1989, UIST '89.

[40]  Alan M. Davis,et al.  Identifying and measuring quality in a software requirements specification , 1993, [1993] Proceedings First International Software Metrics Symposium.

[41]  Antonio C. Siochi,et al.  The UAN: a user-oriented representation for direct manipulation interface designs , 1990, TOIS.

[42]  Antonio C. Siochi,et al.  Task-oriented representation of asynchronous user interfaces , 1989, CHI '89.

[43]  James D. Foley,et al.  Controlling user interface objects through pre- and postconditions , 1992, CHI.

[44]  David A. Carr,et al.  Specification of interface interaction objects , 1994, CHI '94.

[45]  Robert Sedgewick,et al.  Techniques for Algorithm Animation , 1985, IEEE Software.

[46]  Thomas R. G. Green,et al.  The Structure of Command Languages: An Experiment on Task-Action Grammar , 1989, Int. J. Man Mach. Stud..

[47]  Thomas R. G. Green,et al.  When do diagrams make good computer languages , 1979 .

[48]  Ben Shneiderman,et al.  Multiparty Grammars and Related Features for Defining Interactive Systems , 1982, IEEE Transactions on Systems, Man, and Cybernetics.

[49]  David Turo,et al.  Adapting Treemaps to Stock Portfolio Visualization , 1992 .

[50]  Ben Shneiderman,et al.  Tree visualization with tree-maps: 2-d space-filling approach , 1992, TOGS.

[51]  Brad A. Myers,et al.  The importance of pointer variables in constraint models , 1991, UIST '91.

[52]  Steven P. Reiss,et al.  PECAN: Program Development Systems that Support Multiple Views , 1984, IEEE Transactions on Software Engineering.

[53]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[54]  Christopher Andrew Rouff.  Specification and rapid prototyping of user interfaces , 1992 .

[55]  James J. Horning,et al.  Formal specification as a design tool , 1980, POPL '80.