Multi-modal Interaction in Biomedicine

Everybody agrees that user tasks and preferences should play a central role in the design and development of applications oriented to non-computer experts. Nevertheless, even biomedical applications are sometimes developed in a relative vacuum from the real needs of end-users and environments where they are supposed to be used. To provide a clinician with an intuitive environment to solve a target class of problems, a biomedical application has to be built in such a way that a user can exploit modern technologies without specialized knowledge of underlying hardware and software [18]. Unfortunately, in reality the situation is different. Many developers do not take into account the fact that their potential users are people, who are mostly inexperienced computer users, and as a result they need intuitive interaction capabilities and a relevant feedback adapted to their knowledge and skills. User comfort is very important for the success of any software application [13]. But very often we forget that usability problems may arise not only from an ‘uncomfortable' graphical user interface (GUI), but also from a projection modality chosen incorrectly for deploying an interactive environment [16]. Existing projection modalities have not been sufficiently investigated yet in respect to usability factors. Meanwhile, the selection of an appropriate projection modality in accordance with the user's tasks, preferences and personal features might help in building a motivated environment for biomedical purposes. In this chapter we summarize our recent findings related to this research and introduce a new concept of multi-modal interaction based on the combination of virtual reality (VR) and desktop projection modalities within the same system. For the case study of the research we used a biomedical application simulating vascular reconstruction [2,22].

[1]  P. Dykstra.  X11 in virtual environments , 1993, Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium.

[2]  P. Enright,et al.  The 6‐min Walk Test: A Quick Measure of Functional Status in Elderly Adults. , 2003, Chest.

[3]  Phillip Dykstra.  X11 in virtual environments: combining computer interaction methodologies , 1994 .

[4]  Tom Rodden,et al.  Designing Virtual Environments to Support Cooperation in the Real World , 2002, Virtual Reality.

[5]  Andrew S. Forsberg,et al.  Image plane interaction techniques in 3D immersive environments , 1997, SI3D.

[6]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[7]  Peter M. A. Sloot,et al.  Simulated vascular reconstruction in a virtual operating theatre , 2001, CARS.

[8]  James S.T. Yao,et al.  Current Techniques in Modern Vascular Surgery , 2000 .

[9]  Peter M. A. Sloot,et al.  A multi-modal interface for an interactive simulated vascular reconstruction system , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[10]  Robert Allen,et al.  Handbook of Medical Imaging—Processing and Analysis , 2001 .

[11]  Peter M. A. Sloot,et al.  Context analysis to support development of virtual reality applications , 2004, Virtual Reality.

[12]  Jon Trinder,et al.  The Humane Interface: New Directions for Designing Interactive Systems , 2002, Interact. Learn. Environ..

[13]  Chaomei Chen,et al.  Individual differences in virtual environments-introduction and overview , 2000 .

[14]  Larry F. Hodges,et al.  User Interface Constraints for Immersive Virtual Environment Applications , 1995 .

[15]  F. Jolesz,et al.  1996 RSNA Eugene P. Pendergrass New Horizons Lecture. Image-guided procedures and the operating room of the future. , 1997, Radiology.

[16]  Ivan E. Sutherland,et al.  The Ultimate Display , 1965 .

[17]  Jef Raskin,et al.  The Humane Interface: New Directions for Designing Interactive Systems , 2000 .

[18]  Daniel M. Brown,et al.  Review of The humane interface , 2002 .

[19]  Peter M. A. Sloot,et al.  A grid-based interactive visualisation of medical images , 2003 .

[20]  Peter M. A. Sloot,et al.  A Client-Server Engine for Parallel Computation of High-Resolution Planes , 2004, International Conference on Computational Science.

[21]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[22]  Diana E. Forsythe,et al.  “It's Just a Matter of Common Sense”: Ethnography as Invisible Work , 1999, Computer Supported Cooperative Work (CSCW).

[23]  Peter M. A. Sloot,et al.  A first step to a user centered approach to a development of adaptive simulation-visualization complexes , 2002 .

[24]  Stephen Gilmore,et al.  Evaluating the Performance of Skeleton-Based High Level Parallel Programs , 2004, International Conference on Computational Science.

[25]  R. G. Belleman,et al.  Interactive Exploration in Virtual Environments , 2003 .

[26]  Isaac N. Bankman,et al.  Handbook of Medical Imaging. Processing and Analysis , 2002 .

[27]  Luis Serra,et al.  Dextrous virtual work , 1996, CACM.

[28]  Oliver Stefani,et al.  Extending the desktop workplace by a portable virtual reality system , 2006, Int. J. Hum. Comput. Stud..