An MDE-based framework to support the development of Mixed Interactive Systems

In the domain of Human-Computer Interaction (HCI), recent advances in sensors, communication technologies, miniaturization and computing capabilities have led to new and advanced forms of interaction. Among them, Mixed Interactive Systems (MIS), form a class of interactive systems that comprises augmented reality, tangible interfaces and ambient computing; MIS aim to take advantage of physical and digital worlds to promote a more transparent integration of interactive systems with the user's environment. Due to the constant change of technologies and the multiplicity of these interaction forms, specific development approaches have been developed. As a result, numerous taxonomies, frameworks, API and models have emerged, each one covering a specific and limited aspect of the development of MIS.To support a coherent use of these multiple development resources and contribute to the increasing popularity of MIS, we have developed a framework based on Model-Driven Engineering. The goal is to take advantage of Model-Driven Engineering (MDE) standards, methodology and tools to support the manipulation of complementary Domain Specific Languages (DSL), to organize and link the use of different design and implementation resources, and to ensure a rationalized implementation based on design choices.In this paper, we first summarize existing uses of MDE in HCI before focusing on five major benefits MDE can provide in a MIS development context. We then detail which MDE tools and resources support these benefits and thus form the pillars of the success of an MDE-based MIS development approach. Based on this analysis, we introduce our framework, called Guide-Me, and illustrate its use through a case study. This framework includes two design models. Model transformations are also included to link one model to another; as a result the frameworks coverage extends from the earliest design step to a software component-based prototyping platform. A toolset based on the Eclipse Modeling Framework (EMF) that supports the use of the framework is also presented. We finally assess our MDE-based development process for MIS based on the five major MDE benefits for MIS. Five benefits justify the use of MDE for developing Mixed Interactive Systems.A combination of MDE tools is proposed to develop complex interactive systems.MDE approach contributes to the increasing popularity of Mixed Interactive Systems.Multiple design facets of MIS are easily supported by MDE approaches.Early design decisions are strongly anchored within a development approach of MIS.

[1]  Nasa,et al.  A metamodel for the runtime architecture of an interactive system: the UIMS tool developers workshop , 1992, SGCH.

[2]  Mohamed Abid,et al.  Towards a new approach of model-based HCI conception , 2006 .

[3]  Michel Riveill,et al.  Wcomp: a Multi-Design Approach for Prototyping Applications using Heterogeneous Resources , 2006, Seventeenth IEEE International Workshop on Rapid System Prototyping (RSP'06).

[4]  Emmanuel Dubois,et al.  A Design-Oriented Information-Flow Refinement of the ASUR Interaction Model , 2007, EHCI/DS-VIS.

[5]  J. Cunningham History , 2007, The Journal of Hellenic Studies.

[6]  Bernd Brügge,et al.  Design of a component-based augmented reality framework , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[7]  Jean-Sébastien Sottet,et al.  Model-Driven Adaptation for Plastic User Interfaces , 2007, INTERACT.

[8]  Goetz Botterweck,et al.  A model-driven approach to the engineering of multiple user interfaces , 2006, MoDELS'06.

[9]  Dominique L. Scapin,et al.  Usability recommendations in the design of mixed interactive systems , 2009, EICS '09.

[10]  Dominique L. Scapin,et al.  Obstacles and Perspectives for Evaluating Mixed Reality Usability , 2004, MIXER.

[11]  Emmanuel Dubois,et al.  ASUR++: Supporting the design of mobile mixed systems , 2003, Interact. Comput..

[12]  Eigil Samset,et al.  Augmented Reality for Minimally Invasive Surgery: Overview and Some Recent Advances , 2010 .

[13]  Jean Bézivin,et al.  On the unification power of models , 2005, Software & Systems Modeling.

[14]  Orit Shaer,et al.  A specification paradigm for the design and implementation of tangible user interfaces , 2009, TCHI.

[15]  Dimitri Masson,et al.  Graphs of Models for Exploring Design Spaces in the engineering of Human Computer Interaction , 2011 .

[16]  Dominique L. Scapin,et al.  Usability Recommendations for Mixed Interactive Systems: Extraction and Integration in a Design Process , 2013 .

[17]  Vicente Pelechano,et al.  A Model Driven Development Method for Developing Context-Aware Pervasive Systems , 2008, UIC.

[19]  David Garlan,et al.  Acme: an architecture description interchange language , 2010, CASCON.

[20]  Stephen Travis Pope,et al.  A cookbook for using the model-view controller user interface paradigm in Smalltalk-80 , 1988 .

[21]  Dieter Schmalstieg,et al.  APRIL: a high-level framework for creating augmented reality presentations , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[22]  Fabio Paternò,et al.  Design and development of multidevice user interfaces through multiple logical descriptions , 2004, IEEE Transactions on Software Engineering.

[23]  Jean-Marie Favre,et al.  Foundations of Model ( Driven ) ( Reverse ) Engineering Episode I : Story of The Fidus Papyrus and the Solarus , 2004 .

[24]  Jean-Sébastien Sottet,et al.  A Model-Driven Engineering Approach for the Usability of Plastic User Interfaces , 2008, EHCI/DS-VIS.

[25]  Emmanuel Dubois,et al.  ASUR++: A Design Notation for Mobile Mixed Systems , 2002, Mobile HCI.

[26]  Philip R. Cohen,et al.  Creating tangible interfaces by augmenting physical objects with multimodal language , 2001, IUI '01.

[27]  Saul Greenberg,et al.  Phidgets: easy development of physical interfaces through physical widgets , 2001, UIST '01.

[28]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[29]  James D. Foley,et al.  History, Results, and Bibliography of the User Interface Design Environment (UIDE), an Early Model-based System for User Interface Design and Implementation , 1994, DSV-IS.

[30]  Laurence Nigay,et al.  The Engineering of Mixed Reality Systems , 2009, Human-Computer Interaction Series.

[31]  Emmanuel Dubois,et al.  Classification Space for Augmented Surgery, an Augmented Reality Case Study , 1999, INTERACT.

[32]  Christian Sandor,et al.  CUIML: A language for generating multimodal human-computer interfaces , 2001 .

[33]  Jean Vanderdonckt,et al.  A Unifying Reference Framework for multi-target user interfaces , 2003, Interact. Comput..

[34]  Lionel Médini,et al.  Reinventing the familiar: exploring an augmented reality design space for air traffic control , 1998, CHI.

[35]  Cédric Bach,et al.  MACS: combination of a formal mixed interaction model with an informal creative session , 2011, EICS '11.

[36]  Fabio Paternò,et al.  ConcurTaskTrees: An Engineered Notation for Task Models , 2003 .

[37]  Orit Shaer,et al.  Reality-based interaction: a framework for post-WIMP interfaces , 2008, CHI.

[38]  Adrian Freed,et al.  Features and Future of Open Sound Control version 1.1 for NIME , 2009, NIME.

[39]  Wolfgang Broll,et al.  Mixed reality user interface description language , 2006, SIGGRAPH '06.

[40]  Hiroshi Ishii,et al.  Token+constraint systems for tangible interaction with digital information , 2005, TCHI.

[41]  T. C. Nicholas Graham,et al.  An Incremental Algorithm for High-Performance Runtime Model Consistency , 2009, MoDELS.

[42]  Céline Coutrix,et al.  An Integrating Framework for Mixed Systems , 2010, The Engineering of Mixed Reality Systems.

[43]  Scott R. Klemmer,et al.  Papier-Mache: toolkit support for tangible input , 2004, CHI.

[44]  Emmanuel Dubois,et al.  Taking Advantage of Model-Driven Engineering Foundations for Mixed Interaction Design , 2011, Model-Driven Development of Advanced User Interfaces.

[45]  D. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[46]  Donald A. Norman,et al.  User Centered System Design , 1986 .

[47]  Oscar Pastor,et al.  Dealing with Abstract Interaction Modeling in an MDE Development Process: A Pattern-Based Approach , 2007, Interacción.

[48]  Frank Vahid,et al.  Enabling nonexpert construction of basic sensor-based systems , 2009, TCHI.

[49]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).