An infrastructure for realizing custom-tailored augmented reality user interfaces

Augmented reality (AR) technologies are rapidly expanding into new application areas. However, the development of AR user interfaces and appropriate interaction techniques remains a complex and time-consuming task. Starting from scratch is more common than building upon existing solutions. Furthermore, adaptation is difficult, often resulting in poor quality and limited flexibility with regard to user requirements. In order to overcome these problems, we introduce an infrastructure for supporting the development of specific AR interaction techniques and their adaptation to individual user needs. Our approach is threefold: a flexible AR framework providing independence from particular input devices and rendering platforms, an interaction prototyping mechanism allowing for fast prototyping of new interaction techniques, and a high-level user interface description, extending user interface descriptions into the domain of AR. The general usability and applicability of the approach is demonstrated by means of three example AR projects.

[1]  Blair MacIntyre,et al.  DART: a toolkit for rapid design exploration of augmented reality experiences , 2005, ACM Trans. Graph..

[2]  F. Aish,et al.  Augmented Reality Meeting Table: a Novel Multi-User Interface for Architectural Design , 2004 .

[3]  Ralf Dörner,et al.  Efficient Mixed Reality Application Development , 2004 .

[4]  Sofoklis Sotiriou,et al.  Connect: Designing the Classroom of Tomorrow by Using Advanced Technologies to Connect Formal and Informal Learning Environments , 2004, CELDA.

[5]  Christian Sandor,et al.  CUIML: A language for generating multimodal human-computer interfaces , 2001 .

[6]  C. Just,et al.  Software Tools for Virtual Reality Application Development , 1998, SIGGRAPH 1998.

[7]  Werner Hartmann,et al.  Authoring of a mixed reality assembly instructor for hierarchical structures , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[8]  Andreas Butz,et al.  A survey of software infrastructures and frameworks for ubiquitous computing , 2005, Mob. Inf. Syst..

[9]  Deborah Hix,et al.  User-Centered Design and Evaluation of Virtual Environments , 1999, IEEE Computer Graphics and Applications.

[10]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[11]  Dieter Schmalstieg,et al.  Bridging multiple user interface dimensions with augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[12]  Steven K. Feiner,et al.  Unit: modular development of distributed interaction techniques for highly interactive user interfaces , 2004, GRAPHITE '04.

[13]  Gerard Jounghyun Kim,et al.  Immersive authoring of tangible augmented reality applications , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[14]  H. James Hoover,et al.  InTml: a description language for VR applications , 2002, Web3D '02.

[15]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[16]  Dieter Schmalstieg,et al.  An open software architecture for virtual reality interaction , 2001, VRST '01.

[17]  Hirokazu Kato,et al.  A registration method based on texture tracking using ARToolKit , 2003, 2003 IEEE International Augmented Reality Toolkit Workshop.

[18]  Michael Haller,et al.  Authoring of Mixed Reality Applications including Multi-Marker Calibration for Mobile Devices , 2004, EGVE.

[19]  George Papagiannakis,et al.  VHD++ development framework: towards extendible, component based VR/AR simulation engine featuring advanced virtual character technologies , 2003, Proceedings Computer Graphics International 2003.

[20]  Carolina Cruz-Neira,et al.  VR Juggler: a virtual platform for virtual reality application development , 2001, Proceedings IEEE Virtual Reality 2001.

[21]  D. F. Abawi,et al.  CVMP 2004 - efficient mixed reality application development , 2004 .

[22]  T. Alt,et al.  Augmented Reality for manufacturing planning , 2003, IPT/EGVE.

[23]  Daniela Gorski Trevisan,et al.  Exploring the design and engineering of mixed reality systems , 2004, IUI '04.

[24]  Ava Fatah gen. Schieck,et al.  ARTHUR: A Collaborative Augmented Environment for Architectural Design and Urban Planning , 2004, J. Virtual Real. Broadcast..

[25]  Mark Billinghurst,et al.  Augmenting the science centre and museum experience , 2004, GRAPHITE '04.

[26]  Gregg C. Vanderheiden,et al.  Abstract user interface representations: how well do they support universal access? , 2002 .

[27]  Wolfgang Friedrich,et al.  ARVIKA-augmented reality for development, production and service , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[28]  Martin Bauer,et al.  Fata Morgana - a presentation system for product design , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[29]  Dieter Schmalstieg,et al.  APRIL: a high-level framework for creating augmented reality presentations , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[30]  Bernd Brügge,et al.  Design of a component-based augmented reality framework , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[31]  L. Thiele,et al.  The case for reconfigurable hardware in wearable computing , 2003, Personal and Ubiquitous Computing.

[32]  Olaf Oehme,et al.  Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation , 2003, Int. J. Hum. Comput. Interact..

[33]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[34]  Wolfgang Broll,et al.  Interactive Props and Choreography Planning with the Mixed Reality Stage , 2004, ICEC.

[35]  Torsten Fröhlich,et al.  The MORGAN framework: enabling dynamic multi-user AR and VR projects , 2004, VRST '04.

[36]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[37]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[38]  Rainer Simon,et al.  A generic uiml vocabulary for device- and modality independent user interfaces , 2004, WWW Alt. '04.

[39]  Christian Sandor,et al.  Towards a Development Methodology for Augmented Reality User Interfaces , 2004, MIXER.

[40]  Wayne Piekarski,et al.  ARQuake: the outdoor augmented reality gaming system , 2002, CACM.

[41]  Bruce H. Thomas,et al.  An architecture for outdoor wearable computers to support augmented reality and multimedia applications , 1999, 1999 Third International Conference on Knowledge-Based Intelligent Information Engineering Systems. Proceedings (Cat. No.99TH8410).

[42]  Steven K. Feiner,et al.  Augmented Reality in Architectural Construction, Inspection, and Renovation , 1996 .

[43]  Frank Biocca,et al.  Comparative effectiveness of augmented reality in object assembly , 2003, CHI '03.

[44]  Doug A. Bowman,et al.  A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods , 2002, Presence: Teleoperators & Virtual Environments.