A survey of plasticity in 3D user interfaces

Plasticity of 3D user interfaces refers to their capabilities to automatically fit to a set of hardware and environmental constraints. This area of research has already been deeply explored in the domain of traditional 2D user interfaces. Besides, during the last decade, interest in 3D user interfaces has grown. Designers find with 3D user interfaces new ways to promote and interact with data, such as e-commerce websites, scientific data visualization, etc. Because of the wide variety of Virtual Reality (VR) and Augmented Reality (AR) applications in terms of hardware, data and target users, there is a real interest in solutions for automatic adaption in order to improve the user experience in any context while reducing the development costs. An adaptation is performed in reaction to different criteria defining a system such as the targeted hardware platform, the user's context and the structure and the semantic of the manipulated data. This adaptation can then impact the system in different ways, especially content presentation, interaction techniques modifications and eventually the current distribution of the system across a set of available devices. We present the state of the art about plastic 3D user interfaces. Moreover, we present well known methods in the field of 2D user interfaces that could become relevant for 3D user interfaces. With this survey, we show that current solutions do not meet all plasticity requirements. That is why we propose an action plan to meet these requirements.

[1]  John F. Bradley,et al.  An agent-based approach to adaptive navigational support within 3D-environments , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[2]  Supporting user interfaces for all through user modeling , 1995 .

[3]  Naokazu Yokoya,et al.  Geometric and photometric registration for real-time augmented reality , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[4]  Peter Brusilovsky,et al.  Methods and techniques of adaptive hypermedia , 1996, User Modeling and User-Adapted Interaction.

[5]  Dennis Wiebusch,et al.  Enhanced decoupling of components in intelligent realtime interactive systems using ontologies , 2012, 2012 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS).

[6]  Joe Lo,et al.  The Grappl 3D interaction technique library , 2004, VRST '04.

[7]  Russell M. Taylor,et al.  VRPN: a device-independent, network-transparent VR peripheral system , 2001, VRST '01.

[8]  Flavia Sparacino.  Sto(ry)chastics: A Bayesian Network Architecture for User Modeling and Compuatational Storytelling for Interactive Spaces , 2003, UbiComp.

[9]  Augusto Celentano,et al.  Adaptive interaction in Web3D virtual worlds , 2004, Web3D '04.

[10]  Jean Vanderdonckt,et al.  QTk: An Integrated Model-Based Approach to Designing Executable User Interfaces , 2000 .

[11]  Ronan Querrec,et al.  PEGASE: A Generic and Adaptable Intelligent System for Virtual Reality Learning Environments , 2010 .

[12]  Dieter Schmalstieg,et al.  An open software architecture for virtual reality interaction , 2001, VRST '01.

[13]  Guanling Chen,et al.  A Survey of Context-Aware Mobile Computing Research , 2000 .

[14]  Torsten Fröhlich,et al.  The MORGAN framework: enabling dynamic multi-user AR and VR projects , 2004, VRST '04.

[15]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[16]  William E. Lorensen,et al.  Decimation of triangle meshes , 1992, SIGGRAPH.

[17]  Irma Lindt,et al.  Adaptive 3D-User-Interfaces , 2009 .

[18]  Thierry Duval,et al.  A migration mechanism to manage network troubles while interacting within collaborative virtual environments , 2006, VRCIA '06.

[19]  Jérôme Royan,et al.  Real-time Interaction with Complex Models 3d Streaming Requirements Network-based Visualization of 3d Landscapes and City Models , 2022 .

[20]  Ethan Marcotte.  Responsive Web Design , 2011 .

[21]  Samir Otmane,et al.  3D Interaction Assistance in Virtual Reality: A Semantic Reasoning Engine for Context-awareness - From Interests and Objectives Detection to Adaptations , 2013, GRAPP/IVAPP.

[22]  Wojciech Cellary,et al.  Interactive 3D Multimedia Content , 2012, Springer London.

[23]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[24]  Edward Ross,et al.  Intelligent User Interfaces: Survey and Research Directions , 2000 .

[25]  Raimund Dachselt,et al.  Three-dimensional menus: A survey and taxonomy , 2007, Comput. Graph..

[26]  Fernando Santos Osório,et al.  AdapTIVE: an intelligent virtual environment and its application in e-commerce , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[27]  Luca Chittaro,et al.  Dynamic generation of personalized VRML content: a general approach and its application to 3D e-commerce , 2002, Web3D '02.

[28]  Marek Hatala,et al.  Ontology and rule based retrieval of sound objects in augmented audio reality system for museum visitors , 2004, SAC '04.

[29]  Themis Panayiotopoulos,et al.  A platform for virtual museums with personalized content , 2009, Multimedia Tools and Applications.

[30]  Zoltán Fiala,et al.  AMACONT: A System Architecture for Adaptive Multimedia Web Applications , 2004, Berliner XML Tage.

[31]  Raimund Dachselt,et al.  Using the AMACONT architecture for flexible adaptation of 3D web applications , 2006, Web3D '06.

[32]  A. Kobsa.  User Modeling : Recent Work , Prospects and Hazards , 1993 .

[33]  Irma Lindt.  Exchangeability of 3 D Interaction Techniques , 2005 .

[34]  Ipke Wachsmuth,et al.  A User-Adaptive Interface Agency for Interaction with a Virtual Environment , 1995, Adaption and Learning in Multi-Agent Systems.

[35]  Ioan Marius Bilasco,et al.  Semantic-based rules for 3D scene adaptation , 2007, Web3D '07.

[36]  Joëlle Coutaz,et al.  Plasticity of User Interfaces: Framework and Research Agenda , 1999, INTERACT.

[37]  Steven K. Feiner,et al.  Information filtering for mobile augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[38]  Joëlle Coutaz,et al.  Towards a New Generation of Widgets for Supporting Software Plasticity: The "Comet" , 2004, EHCI/DS-VIS.

[39]  Marc Erich Latoschik.  Designing transition networks for multimodal VR-interactions using a markup language , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[40]  Pierre Dragicevic,et al.  Input Device Selection and Interaction Configuration with ICON , 2001, BCS HCI/IHM.

[41]  Peter Brusilovsky,et al.  Adaptive Hypermedia , 2001, User Modeling and User-Adapted Interaction.

[42]  Philippe Gorce,et al.  Interaction techniques for older adults using touchscreen devices: a literature review from 2000 to 2013 , 2017 .

[43]  Gregory M. P. O'Hare,et al.  ENTER: the personalisation and contextualisation of 3-dimensional worlds , 2000, Proceedings 8th Euromicro Workshop on Parallel and Distributed Processing.

[44]  Sisi Zlatanova,et al.  Semantic Enhancement of a Virtual Reality Simulation System for Fire Fighting , 2011 .

[45]  Alfred Kobsa.  User Modeling and User-Adapted Interaction , 2005, User Modeling and User-Adapted Interaction.

[46]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[47]  Bill N. Schilit,et al.  Context-aware computing applications , 1994, Workshop on Mobile Computing Systems and Applications.

[48]  Ronan Querrec,et al.  Semantic modeling of Virtual Environments using MASCARET , 2012, 2012 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS).

[49]  Chris Hand,et al.  A Survey of 3D Interaction Techniques , 1997, Comput. Graph. Forum.

[50]  Wolfgang Broll,et al.  DEVAL - A Device Abstraction Layer for VR/AR , 2007, HCI.

[51]  Mark Green,et al.  A layout framework for 3D user interfaces , 2005, VRST '05.

[52]  Gerd Bruder,et al.  Viargo - A generic virtual reality interaction library , 2012, 2012 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS).

[53]  Jérôme Royan,et al.  A flexible framework to personalize 3D web users experience , 2010, Web3D '10.

[54]  Marc Erich Latoschik.  A user interface framework for multimodal VR interactions , 2005, ICMI '05.

[55]  Karin Coninx,et al.  A Conceptual Framework for Adaptation and Personalization in Virtual Environments , 2009, 2009 20th International Workshop on Database and Expert Systems Application.

[56]  Chris Joslin,et al.  Adaptation mechanism for three dimensional content within the MPEG-21 framework , 2004 .

[57]  Montserrat Sendín,et al.  Plastic User Interfaces: Designing for Change , 2004, MBUI.