Integrating Mid-Air Haptics into Movie Experiences

"Seeing is believing, but feeling is the truth". This idiom from the seventieth century English clergyman Thomas Fuller gains new momentum in light of an increased proliferation of haptic technologies that allow people to have various kinds of `touch' and `touchless' interactions. Here, we report on the process of creating and integrating touchless feedback (i.e. mid-air haptic stimuli) into short movie experiences (i.e. one-minute movie format). Based on a systematic evaluation of user's experiences of those haptically enhanced movies, we show evidence for the positive effect of haptic feedback during the first viewing experience, but also for a repeated viewing after two weeks. This opens up a promising design space for content creators and researchers interested in sensory augmentation of audiovisual content. We discuss our findings and the use of mid-air haptics technologies with respect to its effect on users' emotions, changes in the viewing experience over time, and the effects of synchronisation.

[1]  Martin Faust,et al.  Haptic Feedback in Pervasive Games , 2006 .

[2]  Ali Israr,et al.  Touch, Taste, & Smell User Interfaces: The Future of Multisensory HCI , 2016, CHI Extended Abstracts.

[3]  Heiko Hecht,et al.  The Big Picture: Effects of Surround on Immersion and Size Perception , 2014, Perception.

[4]  Sriram Subramanian,et al.  Emotions Mediated Through Mid-Air Haptics , 2015, CHI.

[5]  Juyun Lim,et al.  Derivation and evaluation of a labeled hedonic scale. , 2009, Chemical senses.

[6]  Ali Israr,et al.  FeelCraft: crafting tactile experiences for media using a feel effect library , 2014, UIST.

[7]  M. Reiner,et al.  Sensory dominance in combinations of audio, visual and haptic stimuli , 2009, Experimental Brain Research.

[8]  Yi-Hsuan Yang,et al.  A Regression Approach to Music Emotion Recognition , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[9]  Sungjae Hwang,et al.  The Haptic steering Wheel: Vibro-tactile based navigation for the driving environment , 2010, 2010 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[10]  B. Bushman,et al.  Immersed in violence: Presence mediates the effect of 3D violent video gameplay on angry feelings. , 2014 .

[11]  Ali Israr,et al.  Feel Effects , 2014, ACM Trans. Appl. Percept..

[12]  Christian Timmerer,et al.  Sensory Experience: Quality of Experience Beyond Audio-Visual , 2014, Quality of Experience.

[13]  Homei Miyashita,et al.  Augmented gustation using electricity , 2011, AH '11.

[14]  Hiroyuki Shinoda,et al.  Ultrasonic emission tactile sensing , 1995 .

[15]  J. Lagopoulos Electrodermal activity , 2007, Acta Neuropsychiatrica.

[16]  Weisi Lin,et al.  Mulsemedia: State of the Art, Perspectives, and Challenges , 2014, TOMM.

[17]  Sriram Subramanian,et al.  Talking about tactile experiences , 2013, CHI.

[18]  Anatole Lécuyer,et al.  Framework for enhancing video viewing experience with haptic effects of motion , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[19]  野間 春生,et al.  Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems 参加報告 , 1997 .

[20]  Ali Israr,et al.  Sensing the future of HCI , 2016, Interactions.

[21]  Sriram Subramanian,et al.  Rendering volumetric haptic shapes in mid-air using ultrasound , 2014, ACM Trans. Graph..

[22]  Paul Lemmens,et al.  A body-conforming tactile jacket to enrich movie viewing , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[23]  Sriram Subramanian,et al.  UltraHaptics: multi-point mid-air haptic feedback for touch surfaces , 2013, UIST.

[24]  Ali Israr,et al.  AIREAL: interactive tactile experiences in free air , 2013, ACM Trans. Graph..

[25]  Seung-Chan Kim,et al.  Surround haptics: tactile feedback for immersive gaming experiences , 2012, CHI Extended Abstracts.

[26]  Gabriel-Miro Muntean,et al.  Olfaction-Enhanced Multimedia , 2016, ACM Comput. Surv..

[27]  D. Keltner,et al.  The communication of emotion via touch. , 2009, Emotion.

[28]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[29]  Gabriel-Miro Muntean,et al.  Quality of experience study for multiple sensorial media delivery , 2014, 2014 International Wireless Communications and Mobile Computing Conference (IWCMC).

[30]  Haibo Li,et al.  Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration , 2008, IEEE Transactions on Multimedia.

[31]  Karon E. MacLean,et al.  Tactile Animation by Direct Manipulation of Grid Displays , 2015, UIST.

[32]  W. Boucsein Electrodermal activity, 2nd ed. , 2012 .

[33]  Ian Oakley,et al.  A tactile glove design and authoring system for immersive multimedia , 2010 .

[34]  Sriram Subramanian,et al.  Perception of ultrasonic haptic feedback on the hand: localisation and apparent motion , 2014, CHI.