Perceptual and Cognitive Factors for Self-Motion Simulation in Virtual Environments: How Can Self-Motion Illusions (“Vection”) Be Utilized?

How can we convincingly simulate observer locomotion through virtual environments without having to allow for full physical observer movement? That is, how can we best utilize multi-modal stimulation to provide the compelling illusion of moving through simulated worlds while reducing the overall simulation effort? This chapter provides a review on the contribution and interaction of visual, auditory, vibrational, and biomechanical cues (e.g., walking) for self-motion perception and simulation in VR. We propose an integrative framework and discuss potential synergistic effects of perceptual and cognitive influences on self-motion perception in VEs. Based on this perspective, we envision a lean-and-elegant approach that utilizes multi-modal self-motion illusions and perceptual-cognitive factors in a synergistic manner to improve perceptual and behavioral effectiveness and reduce the demand for physical (loco-)motion interfaces to a more affordable level.

[1]  J P Landolt,et al.  Circular Vection as a Function of Foreground-Background Relationships , 1987, Perception.

[2]  R. W. Wood,et al.  The 'Haunted Swing' illusion. , 1895 .

[3]  M. Braunstein,et al.  Induced self-motion in central vision. , 1985, Journal of experimental psychology. Human perception and performance.

[4]  B. Riecke,et al.  Cognitiveand higher-level contributions to illusory self-motion ' perception ( " vection " ) Does the possibility ' of actual motion affect vection ? , 2014 .

[5]  Y. Nakajima,et al.  Perceptually Plausible Sounds Facilitate Visually Induced Self-Motion Perception (Vection) , 2012, Perception.

[6]  Bernhard E. Riecke,et al.  Compelling Self-Motion Through Virtual Environments Without Actual Self-Motion – Using Self-Motion Illusions ('Vection') to Improve VR User Experience , 2010 .

[7]  W. Johnson,et al.  Importance of the vestibular system in visually induced nausea and self-vection. , 1999, Journal of vestibular research : equilibrium & orientation.

[8]  A. Berthoz,et al.  Perception of linear horizontal self-motion induced by peripheral vision (linearvection) basic characteristics and visual-vestibular interactions , 1975, Experimental Brain Research.

[9]  J. Dichgans,et al.  Differential effects of central versus peripheral vision on egocentric and exocentric motion perception , 1973, Experimental Brain Research.

[10]  T. Heckmann,et al.  Circular Vection as a Function of the Relative Sizes, Distances, and Positions of Two Competing Visual Displays , 1989, Perception.

[11]  W Bles,et al.  Postural and oculomotor signs in labyrinthine-defective subjects. , 1984, Acta oto-laryngologica. Supplementum.

[12]  F.A.M. van der Steen.  Self-motion perception , 1998 .

[13]  Yuichi Murai,et al.  Story visualization of literary works , 2009, J. Vis..

[14]  S Palmisano,et al.  Perceiving self-motion in depth: The role of stereoscopic motion and changing-size cues , 1996, Perception & psychophysics.

[15]  Bernhard E. Riecke,et al.  Auditory self-motion simulation is facilitated by haptic and vibrational cues suggesting the possibility of actual motion , 2009, TAP.

[16]  Shinji Nakamura.  Effects of stimulus eccentricity on vection reevaluated with a binocularly defined depth , 2008 .

[17]  Heinrich H. Bülthoff,et al.  Towards lean and elegant self-motion simulation in virtual reality , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[18]  I P Howard,et al.  Visually-induced reorientation illusions as a function of age. , 2000, Aviation, space, and environmental medicine.

[19]  B. Nath,et al.  To move or not to move , 2001 .

[20]  F Caniard,et al.  Visually induced linear vection is enhanced by small physical accelerations , 2006 .

[21]  Sarah Nichols,et al.  Health and safety implications of virtual reality: a review of empirical evidence. , 2002, Applied ergonomics.

[22]  Jerrold Prothero,et al.  A Unified Approach to Presence and Motion Sickness , 2003 .

[23]  T. Seno,et al.  The object and background hypothesis for vection , 2009, Vision Research.

[24]  Roy A. Ruddle,et al.  Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues , 2004, Presence: Teleoperators & Virtual Environments.

[25]  G J Andersen,et al.  Perception of self-motion: psychophysical and computational approaches. , 1986, Psychological bulletin.

[26]  Heinrich H. Bülthoff,et al.  Simulating believable forward accelerations on a stewart motion platform , 2010, TAP.

[27]  Ian P. Howard,et al.  Human visual orientation , 1982 .

[28]  P. Cz..  Handbuch der physiologischen Optik , 1896 .

[29]  Stephen Palmisano,et al.  Jitter and Size Effects on Vection are Immune to Experimental Instructions and Demands , 2004, Perception.

[30]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[31]  Shinji Nakamura.  Effects of depth, eccentricity and size of additional static stimulus on visually induced self-motion perception , 2006, Vision Research.

[32]  J. Lackner,et al.  Visual Stimulation Affects the Perception of Voluntary Leg Movements during Walking , 1988, Perception.

[33]  Rik Warren,et al.  Perception And Control Of Self-Motion: Implications For Visual Simulation Of Vehicular Locomotion , 1987 .

[34]  Robert S. Allison,et al.  Vection in depth during treadmill locomotion , 2012 .

[35]  J. Rieser,et al.  Biomechanical versus inertial information: stable individual differences in perception of self-rotation. , 2009, Journal of experimental psychology. Human perception and performance.

[36]  François Jouen,et al.  The Effect of Linear Vection on Manual Aiming at Memorized Directions of Stationary Targets , 1993, Perception.

[37]  Klaus H. Hinrichs,et al.  Real Walking through Virtual Environments by Redirection Techniques , 2009, J. Virtual Real. Broadcast..

[38]  K. E. Money,et al.  Visually-induced tilt during parabolic flights , 2004, Experimental Brain Research.

[39]  I. Giannopulu,et al.  Linear-Vection Chronometry along Spinal and Sagittal Axes in Erect Man , 1998, Perception.

[40]  Anne E. Garing,et al.  Calibration of human locomotion and models of perceptual-motor organization. , 1995, Journal of experimental psychology. Human perception and performance.

[41]  Juno Kim,et al.  Display lag and gain effects on vection experienced by active observers. , 2011, Aviation, space, and environmental medicine.

[42]  Ian P Howard,et al.  The Contribution of Motion, the Visual Frame, and Visual Polarity to Sensations of Body Tilt , 1994, Perception.

[43]  Timothy P. McNamara,et al.  Self-motion illusions (vection) in VR — Are they good for anything? , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[44]  R. B. Post.  Circular Vection is Independent of Stimulus Eccentricity , 1988, Perception.

[45]  Michiteru Kitazaki,et al.  Veridical walking inhibits vection perception , 2010 .

[46]  G. Taga,et al.  Treadmill locomotion captures visual perception of apparent motion , 2008, Experimental Brain Research.

[47]  A. Väljamäe.  Auditorily-induced illusory self-motion: A review , 2009, Brain Research Reviews.

[48]  Krista M. Gigone,et al.  Perception of visual speed while moving. , 2005, Journal of experimental psychology. Human perception and performance.

[49]  Ikuko Shibata,et al.  Self-motion perception from expanding and contracting optical flows overlapped with binocular disparity , 2005, Vision Research.

[50]  F. Bonato,et al.  Simulated viewpoint jitter shakes sensory conflict accounts of vection. , 2011, Seeing and perceiving.

[51]  H. C. Warren.  Sensations of rotation. , 1895 .

[52]  Michael W. Haas,et al.  Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues , 2003 .

[53]  Frederick Bonato,et al.  Natural Visual-Field Features Enhance Vection , 2010, Perception.

[54]  S Palmisano,et al.  Global-Perspective Jitter Improves Vection in Central Vision , 2000, Perception.

[55]  Bernhard E. Riecke.  How far can we get with just visual information? Path integration and spatial updating studies in Virtual Reality [Wie weit kommt man mit visueller Information allein? Pfadintegrations- und spatial updating Studien in Virtueller Realität] , 2003 .

[56]  Bernhard E. Riecke.  Simple user-generated motion cueing can enhance self-motion perception (Vection) in virtual reality , 2006, VRST '06.

[57]  Bernhard E. Riecke,et al.  Top-down and multi-modal influences on self-motion perception in virtual reality , 2005 .

[58]  W. Becker,et al.  Perception of angular displacement without landmarks: evidence for Bayesian fusion of vestibular, optokinetic, podokinesthetic, and cognitive information , 2006, Experimental Brain Research.

[59]  Darren Burke,et al.  Coherent Perspective Jitter Induces Visual Illusions of Self-Motion , 2003, Perception.

[60]  H. Bülthoff,et al.  Circular vection is facilitated by a consistent photorealistic scene , 2003 .

[61]  Heinrich H. Bülthoff,et al.  Measurement of instantaneous perceived self-motion using continuous pointing , 2009, Experimental Brain Research.

[62]  B J Frost,et al.  The effect of visual-vestibular conflict on the latency of steady-state visually induced subjective rotation , 1981, Perception & psychophysics.

[63]  Robert S. Allison,et al.  Illusory scene distortion occurs during perceived self-rotation in roll , 2006, Vision Research.

[64]  W. Becker,et al.  Fusion of vestibular and podokinesthetic information during self-turning towards instructed targets , 2002, Experimental Brain Research.

[65]  Daniel Västfjäll,et al.  Perception of Self-motion and Presence in Auditory Virtual Environments , 2004 .

[66]  F. A. M. Steen,et al.  Induction and impairment of saturated yaw and surge vection , 2000, Perception & psychophysics.

[67]  Shoji Sunaga,et al.  Consistent Air Flow to the Face Facilitates Vection , 2011, Perception.

[68]  Marc O. Ernst,et al.  Integration of vestibular and proprioceptive signals for spatial updating , 2011, Experimental Brain Research.

[69]  Mel Slater,et al.  The Influence of Body Movement on Subjective Presence in Virtual Environments , 1998, Hum. Factors.

[70]  I. Howard,et al.  Effect of Field Size, Head Motion, and Rotational Velocity on Roll Vection and Illusory Self-Tilt in a Tumbling Room , 1999, Perception.

[71]  Thomas A. Furness,et al.  Foreground/Background Manipulations Affect Presence , 1995 .

[72]  Shinji Nakamura.  Additional Oscillation Can Facilitate Visually Induced Self-Motion Perception: The Effects of its Coherence and Amplitude Gradient , 2010, Perception.

[73]  Yuki Yamada,et al.  Emotional Sounds Influence Vertical Vection , 2012, Perception.

[74]  Reinhart Jürgens,et al.  Circular vection during voluntary suppression of optokinetic reflex , 2002, Experimental Brain Research.

[75]  Ernst Mach,et al.  Grundlinien der Lehre von den Bewegungsempfindungen , 1967 .

[76]  G. Johansson.  Studies on Visual Perception of Locomotion , 1977, Perception.

[77]  P. Baudonniere,et al.  Cognitive Effects on Visually Induced Body Motion in Children , 1995, Perception.

[78]  I. Howard,et al.  Circularvection about earth-horizontal axes in bilateral labyrinthine-defective subjects. , 1989, Acta oto-laryngologica.

[79]  Paul DiZio,et al.  Perceived self-motion in two visual contexts: dissociable mechanisms underlie perception. , 2006, Journal of vestibular research : equilibrium & orientation.

[80]  Laura F. Fox,et al.  Self-motion perception during locomotor recalibration: more than meets the eye. , 2005, Journal of experimental psychology. Human perception and performance.

[81]  Heinrich H. Bülthoff,et al.  Enhancing the Visually Induced Self-Motion Illusion (Vection) under Natural Viewing Conditions in Virtual Reality , 2004 .

[82]  A. Wertheim,et al.  Neural noise distorts perceived motion: the special case of the freezing illusion and the Pavard and Berthoz effect , 2007, Experimental Brain Research.

[83]  Margaret D. Nolan,et al.  Vection and simulator sickness. , 1990, Military psychology : the official journal of the Division of Military Psychology, American Psychological Association.

[84]  Johannes Dichgans,et al.  Characteristics of moving visual scenes influencing spatial orientation , 1975, Vision Research.

[85]  Marc M. Sebrechts,et al.  HANDBOOK OF VIRTUAL ENVIRONMENTS , 2014 .

[86]  W. Geoffrey Wright,et al.  Linear vection in virtual environments can be strengthened by discordant inertial input , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[87]  Richard H. Y. So,et al.  Isolating the Effects of Vection and Optokinetic Nystagmus on Optokinetic Rotation-Induced Motion Sickness , 2009, Hum. Factors.

[88]  Michiteru Kitazaki,et al.  Attentional Modulation of Self-Motion Perception , 2003, Perception.

[89]  I P Howard,et al.  Effect of Stationary Objects on Illusory Forward Self-Motion Induced by a Looming Display , 1988, Perception.

[90]  Daniel Västfjäll,et al.  Vibrotactile enhancement of auditory induced self-motion and spatial presence , 2006 .

[91]  Shoji Sunaga,et al.  Inconsistent Locomotion Inhibits Vection , 2011, Perception.

[92]  Kay M. Stanney,et al.  Configural Scoring of Simulator Sickness, Cybersickness and Space Adaptation Syndrome: Similarities and Differences? , 2001 .

[93]  E. Wist,et al.  Foreground and background in dynamic spatial orientation , 1975 .

[94]  Bernhard E. Riecke,et al.  Embedding presence-related terminology in a logical and functional model , 2002 .

[95]  Shoji Sunaga,et al.  Attentional load inhibits vection , 2011, Attention, perception & psychophysics.

[96]  Behrang Keshavarz,et al.  Illusory Self-Motion in Virtual Environments , 2014, Handbook of Virtual Environments, 2nd ed..

[97]  E RieckeBernhard,et al.  Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality , 2009 .

[98]  J. Lackner,et al.  Vertical linear self-motion perception during visual and inertial motion: more than weighted summation of sensory inputs. , 2005, Journal of vestibular research : equilibrium & orientation.

[99]  Daniel Västfjäll,et al.  Auditory Landmarks Enhance Circular Vection in Multimodal Virtual Reality , 2009 .

[100]  Heinrich H. Bülthoff,et al.  Cognitive factors can influence self-motion perception (vection) in virtual reality , 2006, TAP.

[101]  I. Howard,et al.  Vection: The Contributions of Absolute and Relative Visual Motion , 1994, Perception.

[102]  G. Taga,et al.  Treadmill Experience Alters Treadmill Effects on Perceived Visual Motion , 2011, PloS one.

[103]  Stephen Palmisano,et al.  Vection Can Be Induced without Global-Motion Awareness , 2012, Perception.

[104]  H. Wallach,et al.  The role of head movements and vestibular and visual cues in sound localization. , 1940 .

[105]  Bernhard E. Riecke.  Cognitive and higher-level contributions to illusory self-motion perception ("vection"): does the possibility of actual motion affect vection? ([日本基礎心理学会]第27回大会 シンポジウム マルチモーダル感覚情報の時空間統合をめぐって) , 2009 .

[106]  James R. Lackner,et al.  Proprioceptive Adaptation and Aftereffects , 2014, Handbook of Virtual Environments, 2nd ed..

[107]  J E Zacher,et al.  Post-rotatory nystagmus and turning sensations after active and passive turning. , 1998, Journal of vestibular research : equilibrium & orientation.

[108]  B Gillam,et al.  Stimulus Eccentricity and Spatial Frequency Interact to Determine Circular Vection , 1998, Perception.

[109]  J. Rieser.  Access to knowledge of spatial structure at novel points of observation. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[110]  Bernhard E. Riecke,et al.  How far can we get with just visual information?: path integration and spatial updating studies in virtual reality , 2003 .

[111]  Aleksander Väljamäe,et al.  Sound for Multisensory Motion Simulators , 2007 .

[112]  Jack M. Loomis,et al.  Visual perception of egocentric distance in real and virtual environments. , 2003 .

[113]  Stephen Palmisano,et al.  Vertical display oscillation effects on forward vection and simulator sickness. , 2007, Aviation, space, and environmental medicine.

[114]  J. Dichgans,et al.  Visual-Vestibular Interaction: Effects on Self-Motion Perception and Postural Control , 1978 .

[115]  Juno Kim,et al.  Vection in Depth during Consistent and Inconsistent Multisensory Stimulation , 2011, Perception.

[116]  Jennifer L. Campos,et al.  Imagined Self-Motion Differs from Perceived Self-Motion: Evidence from a Novel Continuous Pointing Method , 2009, PloS one.

[117]  C C Presson,et al.  Updating after Rotational and Translational Body Movements: Coordinate Structure of Perspective Space , 1994, Perception.

[118]  Timothy P. McNamara,et al.  Spatialized sound enhances biomechanically-induced self-motion illusion (vection) , 2011, CHI.

[119]  L R Young,et al.  Influence of head orientation on visually induced pitch and roll sensation. , 1975, Aviation, space, and environmental medicine.

[120]  Heinrich H. Bülthoff,et al.  Circular, Linear, and Curvilinear Vection in a Large-screen Virtual Environment with Floor Projection , 2008, 2008 IEEE Virtual Reality Conference.

[121]  T. Brandt,et al.  Arthrokinetic nystagmus and ego-motion sensation , 1977, Experimental Brain Research.

[122]  W. Bles,et al.  Motion sickness: only one provocative conflict? , 1998, Brain Research Bulletin.

[123]  B. Riecke.  Compelling Self-Motion Through Virtual Environments without Actual Self-Motion – Using Self-Motion Illusions (“Vection”) to Improve User Experience in VR , 2011 .

[124]  Kenneth R. Boff,et al.  Sensory processes and perception , 1986 .

[125]  J. Lackner,et al.  Induction of illusory self-rotation and nystagmus by a rotating sound-field. , 1977, Aviation, space, and environmental medicine.

[126]  S. Shimojo,et al.  Critical Role of Foreground Stimuli in Perceiving Visually Induced Self-Motion (Vection) , 1999, Perception.

[127]  W Bles,et al.  Circular vection and human posture I. Does the proprioceptive system play a role? , 1977, Agressologie: revue internationale de physio-biologie et de pharmacologie appliquees aux effets de l'agression.

[128]  T. Seno,et al.  Narcissistic People Cannot Be Moved Easily by Visual Stimulation , 2011, Perception.

[129]  Heinrich H. Bülthoff,et al.  Tricking people into feeling like they are moving when they are not paying attention , 2008, APGV '08.