A rapid prototyping software infrastructure for user interfaces in ubiquitous augmented reality

Recent user interface concepts, such as multimedia, multimodal, wearable, ubiquitous, tangible, or augmented-reality-based (AR) interfaces, each cover different approaches that are all needed to support complex human–computer interaction. Increasingly, an overarching approach towards building what we call ubiquitous augmented reality (UAR) user interfaces that include all of the just mentioned concepts will be required. To this end, we present a user interface architecture that can form a sound basis for combining several of these concepts into complex systems. We explain in this paper the fundamentals of DWARF’s user interface framework (DWARF standing for distributed wearable augmented reality framework) and an implementation of this architecture. Finally, we present several examples that show how the framework can form the basis of prototypical applications.

[1]  Joëlle Coutaz,et al.  A design space for multimodal systems: concurrent processing and data fusion , 1993, INTERCHI.

[2]  Wolfgang Broll,et al.  The virtual round table - a collaborative augmented multi-user environment , 2000, CVE '00.

[3]  Dieter Schmalstieg,et al.  Integrating Studierstube and DWARF , 2003 .

[4]  S. Feiner,et al.  Unit — A modular framework for interaction technique design , development and implementation , 2002 .

[5]  Holger Regenbrecht,et al.  Interaction in a collaborative augmented reality environment , 2002, CHI Extended Abstracts.

[6]  Christian Sandor,et al.  SHEEP: The Shared Environment Entertainment Pasture , 2002 .

[7]  Hiroshi Ishii,et al.  The metaDESK: models and prototypes for tangible user interfaces , 1997, UIST '97.

[8]  Didier Stricker,et al.  An Optically Based Direct Manipulation Interface for Human-Computer Interaction in an Augmented World , 1999, EGVE.

[9]  Paul S. Strauss,et al.  An object-oriented 3D graphics toolkit , 1992, SIGGRAPH.

[10]  Sharon L. Oviatt,et al.  Mutual disambiguation of recognition errors in a multimodel architecture , 1999, CHI '99.

[11]  Friedemann Mattern,et al.  Pervasive/Ubiquitous Computing , 2001, Informatik-Spektrum.

[12]  Steven K. Feiner,et al.  Windows on the world: 2D windows for 3D augmented reality , 1993, UIST '93.

[13]  Jun Rekimoto,et al.  Pick-and-drop: a direct manipulation technique for multiple computer environments , 1997, UIST '97.

[14]  Michael Norman,et al.  Nonlinear interactive motion control techniques for virtual space navigation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[15]  Dan R. Olsen,et al.  User Interface Management Systems: Models and Algorithms , 1991 .

[16]  Steven K. Feiner,et al.  A menu interface for wearable computing , 2002, Proceedings. Sixth International Symposium on Wearable Computers,.

[17]  Debora Weber-Wulff,et al.  Human-computer interaction in the new millennium , 2002, SOEN.

[18]  Jeffrey S. Shell,et al.  Interacting with groups of computers , 2003, Commun. ACM.

[19]  Roel Vertegaal,et al.  Attentive User Interfaces , 2003 .

[20]  Ivan Poupyrev,et al.  The go-go interaction technique: non-linear mapping for direct manipulation in VR , 1996, UIST '96.

[21]  Bernd Brügge,et al.  Distributed user tracking concepts for augmented reality applications , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[22]  Hiroshi Ishii,et al.  Emerging frameworks for tangible user interfaces , 2000, IBM Syst. J..

[23]  Bernd Brügge,et al.  Results of a study on software architectures for augmented reality systems , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[24]  Wolfgang Wahlster,et al.  Readings in Intelligent User Interfaces , 1998 .

[25]  Grzegorz Rozenberg,et al.  High-level Petri Nets: Theory And Application , 1991 .

[26]  M. Weiser,et al.  Hot topics-ubiquitous computing , 1993 .

[27]  G. Klinker,et al.  A fast and robust line-based optical tracker for augmented reality applications , 1999 .

[28]  Ivan Poupyrev,et al.  Tangible Augmented Reality for Human Computer Interaction , 2002 .

[29]  Kurt Jensen.  High-Level Petri Nets , 1982, European Workshop on Applications and Theory of Petri Nets.

[30]  Ken Perlin,et al.  Pad: an alternative approach to the computer interface , 1993, SIGGRAPH.

[31]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[32]  Albrecht Schmidt,et al.  Developing User Interfaces for Wearable Computers: Don't Stop to Point and Click , 2000 .

[33]  Bernd Brügge,et al.  Herding sheep: live system for distributed augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[34]  Robert J. K. Jacob,et al.  A software model and specification language for non-WIMP user interfaces , 1999, TCHI.

[35]  Sharon L. Oviatt,et al.  Ten myths of multimodal interaction , 1999, Commun. ACM.

[36]  Matthew Turk,et al.  Perceptual user interfaces (introduction) , 2000, CACM.

[37]  Ben Shneiderman,et al.  Designing the user interface (videotape) , 1987 .

[38]  Sharon L. Oviatt,et al.  Multimodal interface research: a science without borders , 2000, INTERSPEECH.

[39]  Bernd Brügge,et al.  An architecture concept for ubiquitous computing aware wearable computers , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[40]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[41]  Armando Fox,et al.  Extending tuplespaces for coordination in interactive workspaces , 2004, J. Syst. Softw..

[42]  J. Underkoff.  Urp : A Luminous-Tangible Workbench for Urban Planning and Design , 1999, CHI 1999.

[43]  Dieter Schmalstieg,et al.  OpenTracker-an open software architecture for reconfigurable tracking based on XML , 2001, Proceedings IEEE Virtual Reality 2001.

[44]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[45]  Morten Fjeld,et al.  BUILD-IT: a planning tool for construction and design , 1998, CHI Conference Summary.

[46]  Alexander H. Waibel,et al.  Multimodal interfaces , 1996, Artificial Intelligence Review.

[47]  Martin Wagner,et al.  Herding Sheep: Live System Development for Distributed Augmented Reality , 2003 .

[48]  Steven K. Feiner,et al.  Mutual disambiguation of 3D multimodal interaction in augmented and virtual reality , 2003, ICMI '03.

[49]  Bernd Brügge,et al.  Design of a component-based augmented reality framework , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[50]  Paul A. Beardsley,et al.  Tangible interaction + graphical interpretation: a new approach to 3D modeling , 2000, SIGGRAPH.

[51]  David Garlan,et al.  Project Aura: Toward Distraction-Free Pervasive Computing , 2002, IEEE Pervasive Comput..

[52]  Gudrun Klinker,et al.  The Intelligent Welding Gun: Augmented Reality for Experimental Vehicle Construction , 2004 .

[53]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[54]  Armando Fox,et al.  The Interactive Workspaces Project: Experiences with Ubiquitous Computing Rooms , 2002, IEEE Pervasive Comput..