New Directions in 3D User Interfaces

! Abstract— Three-dimensional user interfaces (3D UIs) support user tasks in many non-traditional interactive systems such as virtual environments and augmented reality. Although 3D UI researchers have been successful in identifying basic user tasks and interaction metaphors, evaluating the usability of 3D interaction techniques, and improving the usability of many applications, 3D UI research now stands at a crossroads. Very few fundamentally new techniques and metaphors for 3D interaction have been discovered in recent years, yet the usability of 3D UIs in many real-world applications is still not at a desirable level. What directions should 3D UI researchers next explore to improve this situation? In this paper, we make some observations about the history of 3D UIs and the current state-of-the-art. Using this evidence, in addition to our own experience, we argue that 3D UI researchers should approach this problem using some new research approaches, which cluster around the concepts of specificity, flavors, implementation, and emerging technologies. We illustrate and discuss some of these new directions using case studies of research projects undertaken in our group. These explorations indicate the promise of these directions for further increasing our understanding of 3D interaction and 3D UI design, and for ensuring the usability of 3D UIs in future applications.

[1]  Frederick P. Brooks.  What's Real About Virtual Reality? , 1999, IEEE Computer Graphics and Applications.

[2]  Steven K. Feiner,et al.  The Flexible Pointer: An Interaction Technique for Selection in Augmented and Virtual Reality , 2003 .

[3]  Randy Pausch,et al.  Virtual reality on a WIM: interactive worlds in miniature , 1995, CHI '95.

[4]  Randy F. Pausch,et al.  Navigation and locomotion in virtual worlds via flight into hand-held miniatures , 1995, SIGGRAPH.

[5]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[6]  Carlo H. Séquin,et al.  Object associations: a simple and practical approach to virtual 3D manipulation , 1995, I3D '95.

[7]  Chris North,et al.  Information-rich virtual environments: theory, tools, and research agenda , 2003, VRST '03.

[8]  Doug A. Bowman,et al.  The Simple Virtual Environment Library: An Extensible Framework for Building VE Applications , 2000, Presence: Teleoperators & Virtual Environments.

[9]  Rudy Darken,et al.  Map usage in virtual environments: orientation issues , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[10]  Yoshifumi Kitamura,et al.  Workshop: Beyond wand and glove based interaction , 2004 .

[11]  Terry Winograd,et al.  Fluid interaction with high-resolution wall-size displays , 2001, UIST '01.

[12]  Daniel C. Robbins,et al.  Three-dimensional widgets , 1992, I3D '92.

[13]  Doug A. Bowman,et al.  Designing Animal Habitats Within an Immersive VE , 1998, IEEE Computer Graphics and Applications.

[14]  Doug A. Bowman,et al.  An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments , 1997, SI3D.

[15]  Shumin Zhai,et al.  Human performance evaluation of manipulation schemes in virtual environments , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[16]  Doug A. Bowman,et al.  Effectiveness of Cloning Techniques for Architectural Virtual Environments , 2006, IEEE Virtual Reality Conference (VR 2006).

[17]  Scott E. Hudson,et al.  Head-tracked orbital viewing: an interaction technique for immersive virtual environments , 1996, UIST '96.

[18]  John F. Lucas.  Design and Evaluation of 3D Multiple Object Selection Techniques , 2005 .

[19]  J. Michael Moshell,et al.  A Two-Handed Interface for Object Manipulation in Virtual Environments , 1995, Presence: Teleoperators & Virtual Environments.

[20]  Ivan Poupyrev,et al.  The go-go interaction technique: non-linear mapping for direct manipulation in VR , 1996, UIST '96.

[21]  Mehdi Setareh,et al.  Virtual-SAP: an immersive tool for visualizing the response of building structures to environmental conditions , 2003, IEEE Virtual Reality, 2003. Proceedings..

[22]  Doug A. Bowman,et al.  Design and evaluation of menu systems for immersive virtual environments , 2001, Proceedings IEEE Virtual Reality 2001.

[23]  Bernd Fröhlich,et al.  The cubic mouse: a new device for three-dimensional input , 2000, CHI.

[24]  Ivan Poupyrev,et al.  An Introduction to 3D User Interface Design , 2001 .

[25]  Daniel Vogel,et al.  Distant freehand pointing and clicking on very large, high resolution displays , 2005, UIST.

[26]  Doug A. Bowman,et al.  Exploring individual differences in raybased selection: strategies and traits , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[27]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[28]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[29]  Mark Green,et al.  JDCAD: A highly interactive 3D modeling system , 1994, Comput. Graph..

[30]  Deborah Hix,et al.  User-Centered Design and Evaluation of Virtual Environments , 1999, IEEE Computer Graphics and Applications.

[31]  Mark R. Mine,et al.  Virtual Environment Interaction Techniques , 1995 .

[32]  Doug A. Bowman,et al.  Interfaces for Cloning in Immersive Virtual Environments , 2004, EGVE.

[33]  Doug A. Bowman,et al.  Resizing beyond widgets: object resizing techniques for immersive virtual environments , 2005, CHI EA '05.

[34]  W. Barfield,et al.  Visualizing the structure of virtual objects using head tracked stereoscopic displays , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[35]  Joseph J. LaViola,et al.  Hands-free multi-scale navigation in virtual environments , 2001, I3D '01.

[36]  Yoshifumi Kitamura,et al.  Virtual object manipulation using dynamically selected constraints with real-time collision detection , 1996, VRST.

[37]  Doug A. Bowman,et al.  The Educational Value of an Information-Rich Virtual Environment , 1999, Presence: Teleoperators & Virtual Environments.

[38]  Robert J. K. Jacob,et al.  A software model and specification language for non-WIMP user interfaces , 1999, TCHI.

[39]  Mark R Mine.  ISAAC: a meta- system for virtual environments , 1997, Comput. Aided Des..

[40]  Michael Gleicher,et al.  The challenges of 3D interaction , 1994, CHI Conference Companion.

[41]  Andrew S. Forsberg,et al.  Image plane interaction techniques in 3D immersive environments , 1997, SI3D.

[42]  Doug A. Bowman,et al.  Overcoming World in Miniature Limitations by a Scaled and Scrolling WIM , 2006, 3D User Interfaces (3DUI'06).

[43]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[44]  Dieter Schmalstieg,et al.  Using transparent props for interaction with the virtual table , 1999, SI3D.

[45]  Patrick Baudisch,et al.  High-Density Cursor: a Visualization Technique that Helps Users Keep Track of Fast-moving Mouse Cursors , 2003, INTERACT.

[46]  Ivan Poupyrev,et al.  A framework and testbed for studying manipulation techniques for immersive VR , 1997, VRST '97.

[47]  Joe Tullio,et al.  Usability analysis of 3D rotation techniques , 1997, UIST '97.

[48]  John T. Kelso,et al.  DIVERSE: A Framework for Building Extensible and Reconfigurable Device-Independent Virtual Environments and Distributed Asynchronous Simulations , 2002, Presence: Teleoperators & Virtual Environments.

[49]  Doug A. Bowman,et al.  Novel Uses of Pinch Gloves™ for Virtual Environment Interaction Techniques , 2002, Virtual Reality.

[50]  Ivan E. Sutherland,et al.  The Ultimate Display , 1965 .

[51]  Doug A. Bowman,et al.  Effects of information layout, screen size, and field of view on user performance in information‐rich virtual environments , 2005, VRST '05.