Bridging multiple user interface dimensions with augmented reality

Studierstube is an experimental user interface system which uses collaborative augmented reality to incorporate true 3D interaction into a productivity environment. This concept is extended to bridge multiple user interface dimensions by including multiple users, multiple host platforms, multiple display types, multiple concurrent applications and a multi-context (i.e. 3D document) interface into a heterogeneous distributed environment. With this architecture, we can explore the user interface design space between pure augmented reality and the popular ubiquitous computing paradigm. We report on our design philosophy, which is centered around the notion of contexts and locales, as well as the underlying software and hardware architecture. Contexts encapsulate a live application together with 3D (visual) and other data, while locales are used to organize geometric reference systems. By separating geometric relationships (locales) from semantic relationships (contexts), we achieve a great amount of flexibility in the configuration of displays. To illustrate our claims, we present several applications, including a cinematographic design tool which showcases many features of our system.

[1]  Dieter Schmalstieg,et al.  Using transparent props for interaction with the virtual table , 1999, SI3D.

[2]  Jun Rekimoto,et al.  A multiple device approach for supporting whiteboard-based interactions , 1998, CHI.

[3]  Mark Weiser.  The computer for the 21st century , 1991 .

[4]  Hiroshi Ishii,et al.  mediaBlocks: physical containers, transports, and controls for online media , 1998, SIGGRAPH.

[5]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[6]  Steven K. Feiner,et al.  A Distributed 3 D Graphics Library , 1998 .

[7]  M. Weiser.  The Computer for the Twenty-First Century , 1991 .

[8]  Mark Billinghurst,et al.  A wearable spatial conferencing space , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[9]  Randy F. Pausch,et al.  Navigation and locomotion in virtual worlds via flight into hand-held miniatures , 1995, SIGGRAPH.

[10]  David Robson,et al.  Smalltalk-80: The Language and Its Implementation , 1983 .

[11]  Michael Gervautz,et al.  The Personal Interaction Panel – a Two‐Handed Interface for Augmented Reality , 1997, Comput. Graph. Forum.

[12]  Paul S. Strauss,et al.  An object-oriented 3D graphics toolkit , 1992, SIGGRAPH.

[13]  Jeffrey Tsao,et al.  CRYSTAL: Building Multicontext Virtual Environments , 1997, Presence: Teleoperators & Virtual Environments.

[14]  Steven K. Feiner,et al.  Enveloping users and computers in a collaborative 3D augmented reality , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[15]  Steven K. Feiner,et al.  Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system , 1999, Comput. Graph..

[16]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[17]  Richard C. Waters,et al.  Locales and beacons: efficient and precise support for large multi-user virtual environments , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[18]  Steven K. Feiner,et al.  A distributed 3D graphics library , 1998, SIGGRAPH.

[19]  Steven K. Feiner,et al.  Worlds within worlds: metaphors for exploring n-dimensional virtual worlds , 1990, UIST '90.

[20]  Jun Rekimoto,et al.  Pick-and-drop: a direct manipulation technique for multiple computer environments , 1997, UIST '97.

[21]  Dieter Schmalstieg,et al.  Distributed Open Inventor: a practical approach to distributed 3D graphics , 1999, VRST '99.