Psychophysics of wearable haptic/tactile perception in a multisensory context

Abstract Multisensory lab based in Peking University, has carried out basic studies in multisensory space and time processing, intersensory binding and haptic / tactile perception. We exploited a typical paradigm of multisensory illusion-temporal ventriloquist effect and applied it in a wide range of multisensory interactions (mainly focused on temporal processing). In this work, we summarized how the tactile stimuli were exploited to compose tactile cues and as tactile apparent motion to interface with other sensory stimuli (visual and auditory stimuli) to examine the underlying perceptual organization in a multisensory context. Moreover, we introduced two examples of wearable haptic/tactile perception in our lab, by using two customized tactile devices and discussed the potential applications in this field.

[1]  Shin'ya Nishida,et al.  Direction of visual apparent motion driven by perceptual organization of cross-modal signals. , 2013, Journal of vision.

[2]  Glenn F. Wilson,et al.  Psychophysiological responses to changes in workload during simulated air traffic control , 1996, Biological Psychology.

[3]  S. Palmer,et al.  A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. , 2012, Psychological bulletin.

[4]  Frank Steinicke,et al.  Stylo and handifact: modulating haptic perception through visualizations for posture training in augmented reality , 2017, SUI.

[5]  Lihan Chen,et al.  The Roles of Attentional Shifts and Attentional Reengagement in Resolving The Spatial Compatibility Effect in Tactile Simon-like Tasks , 2018, Scientific Reports.

[6]  Abdulmotaleb El Saddik,et al.  Spatial resolution of vibrotactile perception on the human forearm when exploiting funneling illusion , 2009, 2009 IEEE International Workshop on Haptic Audio visual Environments and Games.

[7]  F A Mussa-Ivaldi,et al.  Haptic illusions: experiments on human manipulation and perception of "virtual objects". , 1990, Cold Spring Harbor symposia on quantitative biology.

[8]  Sergei Gepshtein,et al.  Stability and change in perception: spatial organization in temporal context , 2004, Experimental Brain Research.

[9]  Zhuanghua Shi,et al.  Interaction of Perceptual Grouping and Crossmodal Temporal Capture in Tactile Apparent-Motion , 2011, PloS one.

[10]  Charles Spence,et al.  The modulation of crossmodal integration by unimodal perceptual grouping: a visuotactile apparent motion study , 2006, Experimental Brain Research.

[11]  Charles Spence,et al.  Can Tactile Stimuli Be Subitised? An Unresolved Controversy within the Literature on Numerosity Judgments , 2008, Perception.

[12]  Kay M. Stanney,et al.  Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations , 2004, IEEE Computer Graphics and Applications.

[13]  J. Taelman,et al.  Textile Integrated Contactless EMG Sensing for Stress Analysis , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[14]  Geraint Rees,et al.  The Cutaneous Rabbit Illusion Affects Human Primary Sensory Cortex Somatotopically , 2006, PLoS biology.

[15]  M. A. Recarte,et al.  Mental workload while driving: effects on visual search, discrimination, and decision making. , 2003, Journal of experimental psychology. Applied.

[16]  Wanjoo Park,et al.  Implicit Emotion Communication , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[17]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[18]  C. Spence,et al.  Numerosity Judgments for Tactile Stimuli Distributed over the Body Surface , 2006, Perception.

[19]  Lihan Chen,et al.  Mutual influences of intermodal visual/tactile apparent motion and auditory motion with uncrossed and crossed arms. , 2013, Multisensory research.

[20]  Avishai Henik,et al.  In touch with the Simon effect. , 2014, Experimental psychology.

[21]  Hermann J. Müller,et al.  Effects of Packet Loss and Latency on the Temporal Discrimination of Visual-Haptic Events , 2010, IEEE Transactions on Haptics.

[22]  Zhuanghua Shi,et al.  What you see depends on what you hear: temporal averaging and crossmodal integration , 2018, bioRxiv.

[23]  Shinsuke Shimojo,et al.  Changing objects lead briefly flashed ones , 2000, Nature Neuroscience.

[24]  Charles Spence,et al.  Multisensory numerosity judgments for visual and tactile stimuli , 2007, Perception & psychophysics.

[25]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[26]  Avishai Henik,et al.  Effects of Numerosity Range on Tactile and Visual Enumeration , 2016, Perception.

[27]  F. A. Geldard,et al.  The Cutaneous "Rabbit": A Perceptual Illusion , 1972, Science.

[28]  Zhuanghua Shi,et al.  Auditory temporal modulation of the visual Ternus effect: the influence of time interval , 2010, Experimental Brain Research.

[29]  H. Strasburger,et al.  Fitting the psychometric function , 1999, Perception & psychophysics.

[30]  Xiaolin Zhou,et al.  Capture of intermodal visual/tactile apparent motion by moving and static sound. , 2011, Seeing and perceiving.

[31]  Georg v. Békésy,et al.  Neural Funneling along the Skin and between the Inner and Outer Hair Cells of the Cochlea , 1959 .

[32]  Romi Nijhawan,et al.  Motion extrapolation in catching , 1994, Nature.

[33]  G. Békésy,et al.  Funneling in the Nervous System and its Role in Loudness and Sensation Intensity on the Skin , 1958 .

[34]  Roger Fan,et al.  Haptics and the heart: Force and tactile feedback system for cardiovascular interventions. , 2018, Cardiovascular revascularization medicine : including molecular interventions.

[35]  Elena Navarro,et al.  Adaptive, Multisensorial, Physiological and Social: The Next Generation of Telerehabilitation Systems , 2018, Front. Neuroinform..

[36]  Zhuanghua Shi,et al.  Influences of intra- and crossmodal grouping on visual and tactile Ternus apparent motion , 2010, Brain Research.

[37]  Lihan Chen,et al.  Tactile flash lag effect: Taps with changing intensities lead briefly flashed taps , 2013, 2013 World Haptics Conference (WHC).

[38]  J. Vroomen,et al.  Intersensory binding across space and time: A tutorial review , 2013, Attention, Perception, & Psychophysics.

[39]  Jon Driver,et al.  Direction of Visual Apparent Motion Driven Solely by Timing of a Static Sound , 2008, Current Biology.

[40]  Avishai Henik,et al.  Tactile enumeration of small quantities using one hand. , 2014, Acta psychologica.

[41]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[42]  Claudio Pacchierotti,et al.  Towards Wearability in Fingertip Haptics: A 3-DoF Wearable Device for Cutaneous Force Feedback , 2013, IEEE Transactions on Haptics.

[43]  P. Jorna Spectral analysis of heart rate and psychological state: A review of its validity as a workload index , 1992, Biological Psychology.

[44]  Jonathan Tong,et al.  Tactile length contraction as Bayesian inference. , 2016, Journal of neurophysiology.

[45]  A. Kramer,et al.  Perceptual organization and focused attention: The role of objects and proximity in visual processing , 1991, Perception & psychophysics.

[46]  F A Geldard,et al.  Saltation in somesthesis. , 1982, Psychological bulletin.

[47]  Charles Spence,et al.  When does visual perceptual grouping affect multisensory integration? , 2004, Cognitive, affective & behavioral neuroscience.

[48]  M. McCloskey,et al.  Somatotopic representation of location: evidence from the Simon effect. , 2014, Journal of experimental psychology. Human perception and performance.

[49]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .

[50]  Vincent Hayward,et al.  Tactile synthesis and perceptual inverse problems seen from the viewpoint of contact mechanics , 2008, TAP.

[51]  Hong Z. Tan,et al.  Short term memory for tactile stimuli , 2008, Brain Research.

[52]  Temuulen Tsagaan Sankey Spatial Resolution , 2008, Encyclopedia of GIS.

[53]  Leonardo Assumpção,et al.  Contextual Cueing of Tactile Search Is Coded in an Anatomical Reference Frame , 2017, Journal of experimental psychology. Human perception and performance.

[54]  J. Vroomen,et al.  Temporal ventriloquism: sound modulates the flash-lag effect. , 2004, Journal of experimental psychology. Human perception and performance.

[55]  Geoff V Merrett,et al.  A narrative review on haptic devices: relating the physiology and psychophysical properties of the hand to devices for rehabilitation in central nervous system disorders , 2013, Disability and rehabilitation. Assistive technology.

[56]  Hong Z. Tan,et al.  When visual transients impair tactile change detection: A novel case of crossmodal change blindness? , 2006, Neuroscience Letters.

[57]  Ryoko Ueoka,et al.  Investigating Haptic Perception of and Physiological Responses to Air Vortex Rings on a User's Cheek , 2017, CHI.

[58]  Leonardo Assumpção,et al.  Contextual cueing: implicit memory of tactile context facilitates tactile search , 2015, Attention, perception & psychophysics.

[59]  Lihan Chen,et al.  Tactile input and empathy modulate the perception of ambiguous biological motion , 2015, Front. Psychol..

[60]  Yong Wang,et al.  iDental: A Haptic-Based Dental Simulator and Its Preliminary User Evaluation , 2012, IEEE Transactions on Haptics.

[61]  Ulrik R. Beierholm,et al.  Causal inference in perception , 2010, Trends in Cognitive Sciences.

[62]  K. Loh,et al.  Micro-patterned graphene-based sensing skins for human physiological monitoring , 2018, Nanotechnology.

[63]  Antonio Frisoli,et al.  Wearable Haptics and Immersive Virtual Reality Rehabilitation Training in Children With Neuromotor Impairments , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[64]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[65]  A. King,et al.  Multisensory integration: perceptual grouping by eye and ear , 2001, Current Biology.

[66]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[67]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.