A multimodal virtual reality interface for 3 D interaction with VTK

The object-oriented visualization Toolkit (VTK) is widely used for scientific visualization. VTK is a visualization library that provides a large number of functions for presenting three-dimensional data. Interaction with the visualized data is controlled with two-dimensional input devices, such as mouse and keyboard. Support for real three-dimensional and multimodal input is non-existent. This paper describes VR-VTK: a multimodal interface to VTK on a virtual environment. Six degree of freedom input devices are used for spatial 3D interaction. They control the 3D widgets that are used to interact with the visualized data. Head tracking is used for camera control. Pedals are used for clutching. Speech input is used for application commands and system control. To address several problems specific for spatial 3D interaction, a number of additional features, such as more complex interaction methods and enhanced depth perception, are discussed. Furthermore, the need for multimodal input to support interaction with the visualization is shown. Two existing VTK applications are ported using VR-VTK to run in a desktop virtual reality system. Informal user experiences are presented.

[1]  Steve Bryson.  Virtual reality in scientific visualization , 1993, Comput. Graph..

[2]  Andrew E. Johnson,et al.  LIMBO/VTK: A Tool for Rapid Tele-Immersive Visualization , 1998 .

[3]  J. C. Mulder,et al.  The personal space station: Bringing interaction within reach , 2002 .

[4]  William Schroeder,et al.  The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics , 1997 .

[5]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[6]  Robert van Liere,et al.  Co-location and tactile feedback for 2D widget manipulation , 2004, IEEE Virtual Reality 2004.

[7]  Anna Vilanova,et al.  DTI visualization with streamsurfaces and evenly-spaced volume seeding , 2004, VISSYM'04.

[8]  Colin Ware,et al.  Eye-hand co-ordination with force feedback , 2000, CHI.

[9]  Chris Shaw,et al.  The decoupled simulation model for virtual reality systems , 1992, CHI '92.

[10]  William E. Lorensen,et al.  The design and implementation of an object-oriented toolkit for 3D graphics and visualization , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[11]  Dennis Proffitt,et al.  Attention and visual feedback: the bimanual frame of reference , 1997, SI3D.

[12]  Donald P. Greenberg,et al.  Perceiving spatial relationships in computer-generated images , 1992, IEEE Computer Graphics and Applications.

[13]  Kellogg S. Booth,et al.  A study of interactive 3D point location in a computer simulated virtual environment , 1997, VRST '97.

[14]  Robert van Liere,et al.  PVR An Architecture for Portable VR Applications , 1999, EGVE.

[15]  Geoffrey S. Hubona,et al.  The relative contributions of stereo, lighting, and background scenes in promoting 3D depth visualization , 1999, TCHI.

[16]  Joseph J. LaViola,et al.  Immersive VR for Scientific Visualization: A Progress Report , 2000, IEEE Computer Graphics and Applications.

[17]  Vladimir Pavlovic,et al.  Speech/Gesture Interface to a Visual-Computing Environment , 2000, IEEE Computer Graphics and Applications.

[18]  Chris Shaw,et al.  Decoupled simulation in virtual reality with the MR toolkit , 1993, TOIS.

[19]  David Robson,et al.  Smalltalk-80: The Language and Its Implementation , 1983 .

[20]  William Ribarsky,et al.  Evaluation of a multimodal interface for 3D terrain visualization , 2002, IEEE Visualization, 2002. VIS 2002..

[21]  Russell M. Taylor,et al.  VRPN: a device-independent, network-transparent VR peripheral system , 2001, VRST '01.

[22]  Robert van Liere,et al.  Interacting with molecular structures: user performance versus system complexity , 2005, EGVE'05.

[23]  Mark Billinghurst,et al.  Put that where? voice and gesture at the graphics interface , 1998, COMG.

[24]  Daniel C. Robbins,et al.  Interactive shadows , 1992, UIST '92.

[25]  Joseph J. LaViola.  Msvt: a virtual reality-based multimodal scientific visualization tool , 1999 .

[26]  Daniel C. Robbins,et al.  Three-dimensional widgets , 1992, I3D '92.