Influence of multi-modality on moving target selection in virtual reality

Abstract Background Owing to recent advances in virtual reality (VR) technologies, effective user interaction with dynamic content in 3D scenes has become a research hotspot. Moving target selection is a basic interactive task in which the user performance research in tasks is significant to user interface design in VR. Different from the existing static target selection studies, the moving target selection in VR is affected by the change in target speed, angle and size, and lack of research on some key factors. Methods This study designs an experimental scenario in which the users play badminton under the condition of VR. By adding seven kinds of modal clues such as vision, audio, haptics, and their combinations, five kinds of moving speed and four kinds of serving angles, and the effect of these factors on the performance and subjective feelings in moving target selection in VR, is studied. Results The results show that the moving speed of the shuttlecock has a significant impact on the user performance. The angle of service has a significant impact on hitting rate, but has no significant impact on the hitting distance. The acquisition of the user performance by the moving target is mainly influenced by vision under the combined modalities; adding additional modalities can improve user performance. Although the hitting distance of the target is increased in the trimodal condition, the hitting rate decreases. Conclusion This study analyses the results of user performance and subjective perception, and then provides suggestions on the combination of modality clues in different scenarios.

[1]  Michael Victor Ilich,et al.  Moving Target Selection in Interactive Video , 2009 .

[2]  Blurred vision , 2007, Nature.

[3]  M A Heller,et al.  Haptic Dominance in Form Perception with Blurred Vision , 1983, Perception.

[4]  Antti Oulasvirta,et al.  Modelling Error Rates in Temporal Pointing , 2016, CHI.

[5]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[6]  Tovi Grossman,et al.  Comet and target ghost: techniques for selecting moving targets , 2011, CHI.

[7]  C. MacKenzie,et al.  Three-Dimensional Movement Trajectories in Fitts' Task: Implications for Control , 1987 .

[8]  Shumin Zhai,et al.  Human on-line response to target expansion , 2003, CHI '03.

[9]  Antonella De Angeli,et al.  Integration and synchronization of input modes during multimodal human-computer interaction , 1997, CHI.

[10]  Adam Faeth,et al.  Emergent effects in multimodal feedback from virtual buttons , 2014, TCHI.

[11]  J. Tresilian.  Hitting a moving target: Perception and action in the timing of rapid interceptions , 2005, Perception & psychophysics.

[12]  David Whitney,et al.  The influence of visual motion on fast reaching movements to a stationary object , 2003, Nature.

[13]  Andy Cockburn,et al.  Multimodal feedback for the acquisition of small targets , 2005, Ergonomics.

[14]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[15]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[16]  I. Scott MacKenzie,et al.  Extending Fitts' law to two-dimensional tasks , 1992, CHI.

[17]  Brian F. G. Katz,et al.  Non-visual identification, localization, and selection of entities of interest in a 3D environment , 2014, Journal on Multimodal User Interfaces.

[18]  J. Krakauer,et al.  Error correction, sensory prediction, and adaptation in motor control. , 2010, Annual review of neuroscience.

[19]  Karin Coninx,et al.  Evaluating the Influence of Multimodal Feedback on Egocentric Selection Metaphors in Virtual Environments , 2006, HAID.

[20]  T. Ichikawa,et al.  Egocentric Object Manipulation in Virtual Environments: Empirical Evaluation of Interaction Techniques , 1998, Comput. Graph. Forum.

[21]  John Anderson,et al.  An evaluation of techniques for selecting moving targets , 2009, CHI Extended Abstracts.

[22]  Martin S. Moran,et al.  A Test of Fitts' Law with Moving Targets , 1980, Human factors.

[23]  H. Pick,et al.  Visual capture produced by prism spectacles , 1965 .

[24]  Carl Gutwin,et al.  The Effects of Feedback on Targeting with Multiple Moving Targets , 2004, Graphics Interface.

[25]  J. Edward Swan,et al.  Toward Disambiguating Multiple Selections for Frustum-Based Pointing , 2006, 3D User Interfaces (3DUI'06).

[26]  Tovi Grossman,et al.  The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area , 2005, CHI.

[27]  Rynson W. H. Lau,et al.  The implicit fan cursor: a velocity dependent area cursor , 2014, CHI.

[28]  Paul Kabbash,et al.  The “prince” technique: Fitts' law and selection using area cursors , 1995, CHI '95.