A Transformation-based Approach to Building Multi-Platform User Interfaces Using a Task Model and the User Interface Markup Language

The widespread emergence of computing devices that go beyond the capabilities of traditional desktop computers has created a challenge for user interface (UI) developers who face the problem of a lack of a unified development process for building these UIs. This dissertation research focuses on creating a simplified development process for building UIs for multiple platforms. As part of this, the necessary building blocks (and their relationships) that can be used in a process to develop multi-platform UIs (MPUIs) are identified and specified. A task model, which is an abstract representation of the tasks that users perform with a system, is used as a high-level platform-independent specification for representing UIs for multiple platforms. The task model is supplemented with additional navigation attributes and containment operators for each target platform to facilitate the UI development process. This contribution is based on the insight that an uncontaminated task model, in conjunction with additional operators, allows different styles of UIs to be derived for different platforms. This development process is evaluated by functional comparison with a few other multi-platform development processes, based on a set of criteria. In particular, a detailed comparison of this approach is performed with the approach used in the TERESA development environment. The process is also evaluated by demonstrating how the new features of this approach allow different styles of UIs to be built not only for a single platform, but also for different platforms. The two underlying notations that are used in this work include the Concurrent Task Tree (CTT) modeling notation for the task model and an intermediate language for UIs, the User Interface Markup Language (UIML). This research associates a new vocabulary with the UIML language to facilitate a multi-step transformation-based MPUI development process.

[1]  T. V. Raman.  Auditory User Interfaces: Toward the Speaking Computer , 1997 .

[2]  David E. Kieras,et al.  The GOMS family of user interface analysis techniques: comparison and contrast , 1996, TCHI.

[3]  H. Rex Hartson,et al.  Temporal Aspects of Tasks in the User Action Notation , 1992, Hum. Comput. Interact..

[4]  Jean Vanderdonckt,et al.  Graceful degradation of user interfaces as a design method for multiplatform systems , 2004, IUI '04.

[5]  James D. Foley,et al.  Model-based user interface design by example and by interview , 1993, UIST '93.

[6]  H. Rex Hartson,et al.  Developing user interfaces: ensuring usability through product & process , 1993 .

[7]  Anneke Kleppe,et al.  MDA explained - the Model Driven Architecture: practice and promise , 2003, Addison Wesley object technology series.

[8]  Manfred Tscheligi,et al.  Design of usable multi-platform interactive systems , 2003, CHI Extended Abstracts.

[9]  Ping Luo,et al.  INT { RCHI ’ 93 24-29 April 1993 MANAGEMENT OF INTERFACE DESIGN IN HUMANOlD , 1997 .

[10]  Kentaro Go,et al.  Scenario-Based Task Analysis , 2003 .

[11]  James D. Foley,et al.  DON: user interface presentation design assistant , 1990, UIST '90.

[12]  David E. Kieras,et al.  Using GOMS for user interface design and evaluation: which technique? , 1996, TCHI.

[13]  Pablo Castells,et al.  Declarative models of presentation , 1997, IUI '97.

[14]  M. Guizani,et al.  A new design methodology for optical hypercube interconnection network , 1995, Proceedings 1st International Conference on Algorithms and Architectures for Parallel Processing.

[15]  Jacob Eisenstein,et al.  Towards a general computational framework for model-based interface development systems , 1998, IUI '99.

[16]  Marc Abrams,et al.  Building Multi-Platform User Interfaces with UIML , 2001, CADUI.

[17]  Jacob Eisenstein,et al.  Towards a general computational framework for model-based interface development systems , 1999, Knowl. Based Syst..

[18]  Manuel A. Pérez-Quiñones,et al.  Using task models to generate multi-platform user interfaces while ensuring usability , 2002, CHI Extended Abstracts.

[19]  Paolo Paolini,et al.  Model-driven development of Web applications: the AutoWeb system , 2000, TOIS.

[20]  Benjamin Michotte,et al.  USIXML: A User Interface Description Language for Context-Sensitive User Interfaces , 2004 .

[21]  Mary Beth Rosson,et al.  The task-artifact cycle , 1991 .

[22]  Mir Farooq Ali.  HTTP: present and future , 1998 .

[23]  Fabio Paternò,et al.  Tool support for designing nomadic applications , 2003, IUI '03.

[24]  Joëlle Coutaz,et al.  A Reference Framework for the Development of Plastic User Interfaces , 2005 .

[25]  Elizabeth D. Mynatt.  Transforming graphical interfaces into auditory interfaces for blind users , 1997 .

[26]  Fabio Paternò.  Model-Based Design and Evaluation of Interactive Applications , 2000 .

[27]  Wolfgang Müller,et al.  A Novel Dialog Model for the Design of Multimodal User Interfaces , 2004, EHCI/DS-VIS.

[28]  Mary Beth Rosson,et al.  Usability Engineering: Scenario-based Development of Human-Computer Interaction , 2001 .

[29]  Joëlle Coutaz,et al.  Embedding Plasticity in the Development Process of Interactive Systems , 2000 .

[30]  Stephen J. Payne,et al.  Task-Action Grammars: A Model of the Mental Representation of Task Languages , 1987, SGCH.

[31]  Steffen Bleul,et al.  Multimodal Dialog Description for Mobile Devices , 2004 .

[32]  Jean Vanderdonckt,et al.  Transformational Development of User Interfaces with Graph Transformations , 2004, CADUI.

[33]  Marc Abrams,et al.  A Multi-Step Process for Generating Multi-Platform User Interfaces using UIML , 2001, ArXiv.

[34]  Kris Luyten,et al.  Dynamic User Interface Generation for Mobile and Embedded Systems with Model-Based User Interface , 2004 .

[35]  Patty Curthoys,et al.  Developing user interfaces: Ensuring usability through product and process , 1997 .

[36]  Gregg C. Vanderheiden,et al.  Abstract representations as a basis for usable user interfaces , 2004, Interact. Comput..

[37]  Brad A. Myers,et al.  User interface software tools , 1995, TCHI.

[38]  David E. Kieras,et al.  Task Analysis and the Design of Functionality , 2014, The Computer Science and Engineering Handbook.

[39]  W. Buxton.  Human-Computer Interaction , 1988, Springer Berlin Heidelberg.

[40]  James D. Foley,et al.  Providing high-level control and expert assistance in the user interface presentation design , 1993, INTERCHI.

[41]  Fabio Paternò,et al.  ConcurTaskTrees: An Engineered Notation for Task Models , 2003 .

[42]  Neel Sundaresan,et al.  Aurora: a conceptual model for Web-content adaptation to support the universal usability of Web-based services , 2000, CUU '00.

[43]  Carole A. Goble,et al.  Teallach: a model-based user interface development environment for object databases , 1999, Proceedings User Interfaces to Data Intensive Systems.

[44]  Fabio Paternò,et al.  A unified method for designing interactive systems adaptable to mobile and stationary platforms , 2003, Interact. Comput..

[45]  Jeffrey Nichols,et al.  Describing Appliance User Interfaces Abstractly with XML , 2004 .

[46]  By MarcAbrams,et al.  UIML : An XML Language for Building Device-Independent User Interfaces , 1999 .

[47]  Paul Francis,et al.  Model Driven Architecture with Executable UML , 2004 .

[48]  Anke Dittmar,et al.  Methodological and Tool Support for a Task-Oriented Development of Interactive Systems , 1999, CADUI.

[49]  Ping Luo,et al.  Beyond interface builders: model-based interface tools , 1993, INTERCHI.

[50]  Henrik Eriksson,et al.  Model-Based Automated Generation of User Interfaces , 1994, AAAI.

[51]  Wolfgang Müller,et al.  Interactive multimodal user interfaces for mobile devices , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[52]  Jean Vanderdonckt,et al.  A Review of XML-compliant User Interface Description Languages , 2003, DSV-IS.

[53]  Jean Vanderdonckt,et al.  Computer-Aided Design of User Interfaces III , 2002, Springer Netherlands.

[54]  T.M. Duffy,et al.  Scenario-Based Design: Envisioning Work and Technology in System Development [Book Review] , 1996, IEEE Transactions on Professional Communication.

[55]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[56]  Kris Luyten,et al.  UIML.NET: an Open UIML Renderer for the .Net Framework , 2004, CADUI.

[57]  Antonio C. Siochi,et al.  The UAN: a user-oriented representation for direct manipulation interface designs , 1990, TOIS.

[58]  David E. Kieras,et al.  Towards a Practical GOMS Model Methodology for User Interface Design , 1988 .

[59]  Jean Vanderdonckt,et al.  KnowiXML: a knowledge-based system generating multiple abstract user interfaces in USIXML , 2004, TAMODIA '04.

[60]  Kris Luyten,et al.  Multi-device Layout Management for Mobile Computing Devices , 2003 .

[61]  Jacob Eisenstein,et al.  Adaptation in automated user-interface design , 2000, IUI '00.

[62]  Paulo Pinheiro da Silva,et al.  User interface declarative models and development environments: a survey , 2000 .

[63]  Dan R. Olsen,et al.  A Programming Language Basis for User Interface Management , 1989 .

[64]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[65]  David Kieras,et al.  GOMS Models for Task Analysis , 2003 .

[66]  Fabio Paternò,et al.  Design and development of multidevice user interfaces through multiple logical descriptions , 2004, IEEE Transactions on Software Engineering.

[67]  Jean Vanderdonckt,et al.  USIXML: A User Interface Description Language Supporting Multiple Levels of Independence , 2004, ICWE Workshops.

[68]  Brad A. Myers,et al.  Automatic, look-and-feel independent dialog creation for graphical user interfaces , 1990, CHI '90.

[69]  Piero Fraternali,et al.  Tools and approaches for developing data-intensive Web applications: a survey , 1999, CSUR.

[70]  H B TIMMERMAN,et al.  What is task analysis? , 1951, Bulletin of the Medical Library Association.

[71]  Edward A. Fox,et al.  An XML Log Standard and Tool for Digital Library Logging Analysis , 2002, ECDL.

[72]  Jacob Eisenstein,et al.  XIML: A Multiple User Interface Representation Framework for Industry , 2005 .

[73]  Charles Wiecha,et al.  ITS: a tool for rapidly developing interactive applications , 1990, TOIS.

[74]  Paulo Pinheiro,et al.  User Interface Declarative Models and Development Environments: A Survey , 2000, DSV-IS.

[75]  Fabio Paternò,et al.  Supporting Interactions with Multiple Platforms Through User and Task Models , 2005 .

[76]  Jean Vanderdonckt,et al.  A Unifying Reference Framework for multi-target user interfaces , 2003, Interact. Comput..

[77]  Jacob Eisenstein,et al.  XIML: a common representation for interaction data , 2002, IUI '02.

[78]  Jeffrey Nichols,et al.  Personal universal controllers: controlling complex appliances with GUIs and speech , 2003, CHI Extended Abstracts.

[79]  Murray Crease,et al.  Using non-speech sounds in mobile computing devices. , 2000 .

[80]  Stefano Ceri,et al.  Web Modeling Language (WebML): a modeling language for designing Web sites , 2000, Comput. Networks.

[81]  Pablo Castells,et al.  Declarative interface models for user interface construction tools: the MASTERMIND approach , 1995, EHCI.

[82]  Jean Vanderdonckt,et al.  Applying model-based techniques to the development of UIs for mobile computers , 2001, IUI '01.

[83]  Egbert Schlungbaum,et al.  Model-based User Interface Software Tools - Current State of Declarative Models , 1996 .

[84]  John M. Carroll,et al.  Making Use: Scenario-Based Design of Human-Computer Interactions , 2000 .

[85]  Hironobu Takagi,et al.  Annotation-based transcoding for nonvisual web access , 2000, Assets '00.

[86]  Hironobu Takagi,et al.  Transcoding proxy for nonvisual web access , 2000, Assets '00.

[87]  H. Johnson,et al.  Task knowledge structures: Psychological basis and integration into system design ☆ , 1991 .

[88]  Gregg C. Vanderheiden,et al.  Abstract user interface representations: how well do they support universal access? , 2002 .

[89]  Fabio Giannetti,et al.  Device Independence and the Web , 2002, IEEE Internet Comput..

[90]  Brad A. Myers,et al.  Past, Present and Future of User Interface Software Tools , 2000, TCHI.

[91]  Jean Vanderdonckt,et al.  Synchronized model-based design of multiple user interfaces , 2001 .

[92]  Kris Luyten,et al.  Generating Context-Sensitive Multiple Device Interfaces from Design , 2004, CADUI.

[93]  Shoji Kurakake,et al.  Roam, a seamless application framework , 2004, J. Syst. Softw..

[94]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[95]  Kouichi Ono,et al.  Annotation-based Web content transcoding , 2000, Comput. Networks.

[96]  Ed Seidewitz,et al.  What Models Mean , 2003, IEEE Softw..

[97]  Peter Johnson,et al.  Rapid prototyping of user interfaces driven by task models , 1995 .

[98]  John Grundy,et al.  AUIT: Adaptable User Interface Technology, with Extended Java Server Pages , 2005 .

[99]  Jean Vanderdonckt,et al.  Towards a Systematic Building of Software Architectures: the TRIDENT Methodological Guide , 1995, DSV-IS.

[100]  Jeffrey Nichols,et al.  Generating remote control interfaces for complex appliances , 2002, UIST '02.

[101]  Pedro A. Szekely.  Retrospective and Challenges for Model-Based Interface Development , 1996, DSV-IS.

[102]  Jean Vanderdonckt,et al.  Addressing the mapping problem in user interface design with UsiXML , 2004, TAMODIA '04.

[103]  Fabio Paternò,et al.  Automatic Support for Usability Evaluation , 1998, IEEE Trans. Software Eng..

[104]  Kai Richter.  Generic Interface Descriptions using XML , 2002, CADUI.

[105]  Norman W. Paton,et al.  The Teallach Tool: Using Models for Flexible User Interface Design , 1999, CADUI.

[106]  Marc Abrams,et al.  UIML: An Appliance-Independent XML User Interface Language , 1999, Comput. Networks.

[107]  Philip J. Barnard,et al.  THE HANDBOOK OF TASK ANALYSIS FOR HUMAN-COMPUTER INTERACTION , 2003 .

[108]  Ahmed Seffah,et al.  Multiple user interfaces : cross-platform applications and context-aware interfaces , 2005 .

[109]  Bonnie E. John,et al.  CPM-GOMS: an analysis method for tasks with parallel activities , 1995, CHI '95.

[110]  Fabio Paternò,et al.  One Model, Many Interfaces , 2002, CADUI.

[111]  Quentin Limbourg.  Multi-Path Development of User Interfaces , 2004 .

[112]  Brad A. Myers,et al.  Model-based user interfaces: what are they and why should we care? , 1994, UIST '94.

[113]  James D. Foley,et al.  Supporting adaptive interfaces in a knowledge-based user interface environment , 1993, IUI '93.

[114]  Peter Johnson,et al.  Human computer interaction: Psychology, task analysis, and software engineering , 1992 .

[115]  Stefano Ceri,et al.  Building Multi-device, Content-Centric Applications Using WebML and the W3I3 Tool Suite , 2000, ER.

[116]  Anke Dittmar,et al.  Adaptive Task Modelling: From Formal Models to XML Representations , 2005 .

[117]  Joëlle Coutaz,et al.  Plasticity of User Interfaces: Framework and Research Agenda , 1999, INTERACT.

[118]  Elizabeth D. Mynatt.  Transforming Graphical Interfaces Into Auditory Interfaces for Blind Users , 1997, Hum. Comput. Interact..

[119]  Debora Weber-Wulff,et al.  Human-computer interaction in the new millennium , 2002, SOEN.

[120]  Mahmoud Naghshineh,et al.  WebSplitter: a unified XML framework for multi-device collaborative Web browsing , 2000, CSCW '00.

[121]  Fabio Paternò,et al.  Flexible interface migration , 2004, IUI '04.

[122]  Joëlle Coutaz,et al.  A Development Process for Plastic User Interfaces , 2003 .

[123]  Fabio Paternò,et al.  CTTE: an environment for analysis and development of task models of cooperative applications , 2001, CHI Extended Abstracts.

[124]  Fabio Paternò,et al.  The TERESA XML Language for the Description of Interactive Systems at Multiple Abstraction Levels , 2004 .

[125]  K. D. Duncan,et al.  TASK ANALYSIS AND TRAINING DESIGN. , 1967 .