Non-visual identification, localization, and selection of entities of interest in a 3D environment

This paper addresses the use of audio and haptics as a mean to reduce the load of the visual channel in interaction tasks within virtual environments. An examination is made regarding the exploitation of audio and/or haptic interactions for the acquisition of a target of interest in an environment containing multiple and obscured distractors. A first study compares means for identifying and locating a specified target among others employing either audio, haptic, or both sensori-motor channels activated simultaneously. Following an analysis of the results and subject comments, an improved multimodal approach is proposed and evaluated in a second study, combining advantages offered by each sensory channel. Results confirm the efficiency and effectiveness of the proposed multimodal approach.

[1]  Maria C. Yang,et al.  Haptic Force-Feedback Devices for the Office Computer: Performance and Musculoskeletal Loading Issues , 2001, Hum. Factors.

[2]  M. Sheelagh T. Carpendale,et al.  Extending Distortion Viewing from 2D to 3D , 1997, IEEE Computer Graphics and Applications.

[3]  Bob-Antoine J. Menelas,et al.  Virtual Reality Technologies (Visual, Haptics, and Audio) in Large Datasets Analysis , 2014 .

[4]  Mark Wright,et al.  The Effect of Haptic Feedback and Stereo Graphics in a 3D Target Acquisition Task , 2002 .

[5]  H S Colburn,et al.  Speech intelligibility and localization in a multi-source environment. , 1999, The Journal of the Acoustical Society of America.

[6]  Michitaka Hirose,et al.  Desk-sized immersive workplace using force feedback grid interface , 2002, Proceedings IEEE Virtual Reality 2002.

[7]  Tovi Grossman,et al.  Multimodal selection techniques for dense and occluded 3D virtual environments , 2009, Int. J. Hum. Comput. Stud..

[8]  Mehdi Ammi,et al.  Haptical exploration of an unsteady flow , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[9]  Brian F. G. Katz,et al.  Audio haptic feedbacks for an acquisition task in a multi-target context , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[10]  Helge Ritter,et al.  Listen to your Data: Model-Based Sonification for Data Analysis , 1999 .

[11]  B. Katz,et al.  ITD INTERPOLATION AND PERSONALIZATION FOR BINAURAL SYNTHESIS USING SPHERICAL HARMONICS , 2012 .

[12]  D. Freides,et al.  Human information processing and sensory modality: cross-modal functions, information complexity, memory, and deficit. , 1974, Psychological bulletin.

[13]  David Swapp,et al.  Interaction with co-located haptic feedback in virtual reality , 2006, Virtual Reality.

[14]  Patrick Langdon,et al.  Multiple haptic targets for motion-impaired computer users , 2003, CHI '03.

[15]  Jirí Sochor,et al.  Manipulating Objects Behind Obstacles , 2007, HCI.

[16]  Martin J.-D. Otis,et al.  Design of a serious game for learning vibrotactile messages , 2012, 2012 IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2012) Proceedings.

[17]  Douglas S. Brungart,et al.  Sound Localization in Multisource Environments , 2009 .

[18]  Seung-Chan Kim,et al.  Haptic and Sound Grid for Enhanced Positioning in a 3-D Virtual Environment , 2007, HAID.

[19]  Gaëtan Parseihian,et al.  Perceptually based head-related transfer function database optimization. , 2012, The Journal of the Acoustical Society of America.

[20]  M. Denis,et al.  Structural properties of spatial representations in blind people: Scanning images constructed from haptic exploration or from locomotion in a 3-D audio virtual environment , 2010, Memory & cognition.

[21]  Karin Coninx,et al.  Evaluating the Influence of Multimodal Feedback on Egocentric Selection Metaphors in Virtual Environments , 2006, HAID.

[22]  Jack T. Dennerlein,et al.  Perceived Musculoskeletal Loading during use of a Force-Feedback Computer Mouse , 1999 .

[23]  Lorna M. Brown,et al.  A first investigation into the effectiveness of Tactons , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[24]  Ian Oakley,et al.  Solving multi-target haptic problems in menu interaction , 2001, CHI Extended Abstracts.

[25]  Bob-Antoine Jerry Ménélas.  Interactive analysis of cavity-flows in a virtual environment , 2012, SCCG.

[26]  Ian Oakley,et al.  Putting the feel in ’look and feel‘ , 2000, CHI.

[27]  Scott E. Hudson,et al.  The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory , 2009, Pervasive.

[28]  Niklas Elmqvist,et al.  View projection animation for occlusion reduction , 2006, AVI '06.

[29]  Karon E. MacLean,et al.  Perceptual Design of Haptic Icons , 2003 .

[30]  Karon E. MacLean,et al.  The hapticon editor: a tool in support of haptic communication research , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[31]  F. Frances Yao,et al.  Computational Geometry , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[32]  Mehdi Ammi,et al.  Audio-haptic intermodal coupling for comparative search tasks , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[33]  Mehdi Ammi,et al.  Force model for CAD selection , 2008, VRST '08.

[34]  Michael Friis Sørensen,et al.  Head-Related Transfer Functions of Human Subjects , 1995 .

[35]  Mehdi Ammi,et al.  A Flexible Method for Haptic Rendering of Isosurface from Volumetric Data , 2008, EuroHaptics.

[36]  Mehdi Ammi,et al.  Design of haptic stimuli for audio-haptic concurrent coupling , 2011, 2011 IEEE International Workshop on Haptic Audio Visual Environments and Games.

[37]  Michel Denis,et al.  NAVIG: augmented reality guidance system for the visually impaired , 2012, Virtual Reality.

[38]  Charlotte Magnusson,et al.  Non Visual Haptic Audio Tools for Virtual Environments , 2006, HAID.

[39]  Andy Cockburn,et al.  Multimodal feedback for the acquisition of small targets , 2005, Ergonomics.

[40]  Brian F. G. Katz,et al.  Evaluation of a Haptic/Audio System for 3-D Targeting Tasks , 2010 .

[41]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .