Modelling Direct Manipulation Using Referent and Statecharts

An approach to modelling mouse gestures in direct manipulation interfaces is presented. The Referent and Statechart languages are used for modelling structure and behaviour, respectively. A gesture is divided into a series of steps, driven by user action and aided by feedback, all modelled as Statechart states. The Referent describes dynamic relations between these states. The states and transitions determine the gesture syntax and the actual relations established and changed during recognition. We discuss how reasoning about these models can support the design of composite gesture recognisers and provide consistency checking and design critique.

[1]  Jonathan Grudin,et al.  Human Computer Interaction: The Year 2000 and Beyond , 1995, HCI.

[2]  Roger B. Dannenberg,et al.  Garnet: comprehensive support for graphical, highly interactive user interfaces , 1990, Computer.

[3]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[4]  Gerhard Fischer,et al.  Embedding computer-based critics in the contexts of design , 1993, INTERCHI.

[5]  William Buxton,et al.  A three-state model of graphical input , 1990, INTERACT.

[6]  Ramana Rao,et al.  A guided tour of the Common Lisp interface manager , 1990, LIPO.

[7]  Panagiotis Markopoulos,et al.  A compositional model for the formal specification of user interface software , 2013 .

[8]  Jean Vanderdonckt,et al.  Computer-Aided Design of User Interfaces II , 1999, Springer Netherlands.

[9]  Brad A. Myers,et al.  State of the Art in User Interface Software Tools , 1992 .

[10]  Niels Vejrup Carlsen,et al.  Modelling user interface software , 1991 .

[11]  William Buxton,et al.  A taxonomy of See-Through Tools , 1995 .

[12]  Pedro A. Szekely.  Retrospective and Challenges for Model-Based Interface Development , 1996, DSV-IS.

[13]  Pierre D. Wellner,et al.  Statemaster: A UIMS based on statecharts for prototyping and target implementation , 1989, CHI '89.

[14]  David Chenho Kung,et al.  Information Systems Engineering , 1993, Springer Berlin Heidelberg.

[15]  Ronald M. Baecker,et al.  Readings in human-computer interaction : toward the year 2000 , 1995 .

[16]  Derek Coleman,et al.  Introducing Objectcharts or how to use Statecharts in object-oriented design , 1992 .

[17]  Brad A. Myers.  A new model for handling input , 1990, TOIS.

[18]  Jeffrey J. P. Tsai,et al.  A noninvasive architecture to monitor real-time distributed systems , 1990, Computer.

[19]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..