A Haptic System for Switching Wind Temperatures Based on Ultrasonic Vibrations, Peltier Elements, and Electrical Resistances for Multisensory Applications

Adding sensory effects to applications has proven to increase users' quality of experience (QoE). As immersive technologies advance in the fields of entertainment, education, training, and so forth, more fine-grained functions are craved by users to have truly captivating and (nearly) realistic experiences. These new functions, in turn, push forward both the industry and academia on developing solutions to overcome the current limitations. This is indeed the particular case of multimedia systems enriched with thermal wind sensations. Whilst both wind and thermal effects have been included separately in such systems, the lack of wind at a low or high temperature has been noticed by users. It is not trivial building, controlling, and working with devices at different temperatures hit by wind at the same time. This is exacerbated by the fact that different technologies are used to this end. In this paper, we introduce a haptic system that allows the creation and delivery of hot and cold airflow synchronised with audiovisual content. The system combines the vaporisation of cooled water by Peltier plates for cold wind and hair dryers for warm wind steered by wind fans. To assess our system, we analysed the time our devices take to deliver these effects considering different starting temperatures in the environment. Furthermore, we annotated three videos with hot wind, cold wind, and both effects respectively to observe the feasibility of having those effects integrated with audiovisual content. Our findings suggest recommendations for the appropriate triggering of these types of sensory effects combined with multimedia content taking into account our setup.

[1]  Celso A. S. Santos,et al.  PlaySEM: a Platform for Rendering MulSeMedia Compatible with MPEG-V , 2015, WebMedia.

[2]  Robert W. Lindeman,et al.  Haptic ChairIO: A system to study the effect of wind and floor vibration feedback on spatial orientation in VEs , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[3]  Christian Timmerer,et al.  An end-to-end tool chain for Sensory Experience based on MPEG-V , 2013, Signal Process. Image Commun..

[4]  Max Rheiner Birdly an attempt to fly , 2014, SIGGRAPH '14.

[5]  Osamu Oshiro,et al.  Non-contact Cold Thermal Display by Controlling Low-temperature Air Flow Generated with Vortex Tube , 2019, 2019 IEEE World Haptics Conference (WHC).

[6]  Simone Diniz Junqueira Barbosa,et al.  Subjective Evaluation of 360-degree Sensory Experiences , 2019, 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP).

[7]  Nigel Williams,et al.  The heat is on , 2003, Current Biology.

[8]  Celso A. S. Santos,et al.  360° Mulsemedia: A Way to Improve Subjective QoE in 360° Videos , 2019, ACM Multimedia.

[9]  Martin Halvey,et al.  Some like it hot: thermal feedback for mobile devices , 2011, CHI.

[10]  José Vasconcelos-Raposo,et al.  The Impact of Olfactory and Wind Stimuli on 360 Videos Using Head-mounted Displays , 2020, ACM Trans. Appl. Percept..

[11]  Gerard Jounghyun Kim,et al.  Design and evaluation of a wind display for virtual reality , 2004, VRST '04.

[12]  Celso A. S. Santos,et al.  MulseOnto: a Reference Ontology to Support the Design of Mulsemedia Systems , 2019, J. Univers. Comput. Sci..

[13]  Jas Brooks,et al.  Trigeminal-based Temperature Illusions , 2020, CHI.

[14]  Ernst Kruijff,et al.  FaceHaptics: Robot Arm based Versatile Facial Haptics for Immersive Environments , 2020, CHI.

[15]  Celso A. S. Santos,et al.  Mulsemedia DIY: A Survey of Devices and a Tutorial for Building Your Own Mulsemedia Environment , 2019, ACM Comput. Surv..

[16]  Estêvão Bissoli Saleme,et al.  A conceptual architecture and a framework for dealing with variability in mulsemedia systems , 2019, ACM SIGMultimedia Records.

[17]  Yi-Ping Hung,et al.  Hapmosphere: Simulating the Weathers for Walking Around in Immersive Environment with Haptics Feedback , 2019, 2019 IEEE World Haptics Conference (WHC).

[18]  Celso A. S. Santos,et al.  A mulsemedia framework for delivering sensory effects to heterogeneous systems , 2019, Multimedia Systems.

[19]  Yasutoshi Makino,et al.  Remote Cooling Sensation Presentation Controlling Mist in Midair , 2020, 2020 IEEE/SICE International Symposium on System Integration (SII).

[20]  Christian Timmerer,et al.  Sensory effect dataset and test setups , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[21]  Yang-Sheng Chen,et al.  Haptic around: multiple tactile sensations for immersive environment and interaction in virtual reality , 2018, VRST.

[22]  Kening Zhu,et al.  Embodied Weather: Promoting Public Understanding of Extreme Weather Through Immersive Multi-Sensory Virtual Reality , 2019, VRCAI.

[23]  Celso A. S. Santos,et al.  Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience , 2020, IEEE MultiMedia.

[24]  Débora C. Muchaluat-Saade,et al.  Semi-automatic synchronization of sensory effects in mulsemedia authoring tools , 2019, WebMedia.

[25]  Tek-Jin Nam,et al.  Thermal Interaction with a Voice-based Intelligent Agent , 2018, CHI Extended Abstracts.

[26]  Christian Timmerer,et al.  Investigating the impact of sensory effects on the Quality of Experience and emotional response in web videos , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[27]  Lynette A. Jones,et al.  The psychophysics of temperature perception and thermal-interface design , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.