Portable Device for Touch, Taste and Smell Sensations in Augmented Reality Experiences

The purpose of this work is to present a portable hardware device that can provide touch, taste and smell sensations to an augmented reality experience. The proposed hardware is part of a mobile five senses augmented reality system for Museums, to improve and augment, as much as possible, the visiting of a museum, i.e., see, ear, touch, feel and experience all its interesting objects. The existing solutions related to the augment of sensing experiences consist of big hardware systems and they are far from being portable. In this work, a new small and portable device is presented, to integrate and connect with the user’s smartphone to provide the complete five-sense experience. The implemented device adds to the complete augmented system, the touch, smell and taste experiences. Moreover, the device is flexible enough to adapt itself to different sizes of the user’s smartphone or tablet. It is powered by a rechargeable battery, which gives the module the ability to keep the system running during the visit of the museum. The core unit is a microcontroller, it receives instructions from the mobile application in the user’s smartphone and acts accordingly, controlling the remaining hardware of the portable device to deliver the five-sense experience to the user. The communication between the device and the mobile application is possible through wireless communication, using a Bluetooth interface. The communication with the remaining module, denoted here as physical interfaces, will be wired. Preliminary results of device’s first prototype are also presented in this paper.

[1]  Ryohei Nakatsu,et al.  Tongue Mounted Interface for Digitally Actuating the Sense of Taste , 2012, 2012 16th International Symposium on Wearable Computers.

[2]  Seung-Chan Kim,et al.  Tactile feedback on flat surfaces for the visually impaired , 2012, CHI Extended Abstracts.

[3]  Yuriko Suzuki,et al.  Air jet driven force feedback in virtual reality , 2005, IEEE Computer Graphics and Applications.

[4]  Takamichi Nakamoto,et al.  Tiny Olfactory Display Using Surface Acoustic Wave Device and Micropumps for Wearable Applications , 2016, IEEE Sensors Journal.

[5]  Jiaqing Lin,et al.  A Handy Image Explorer with Tactile Feedback for Diagnostic Imaging , 2012, 2012 Seventh International Conference on Broadband, Wireless Computing, Communication and Applications.

[6]  Adrian David Cheok,et al.  Electronic taste stimulation , 2011, UbiComp '11.

[7]  Takashi Maeno,et al.  Multi-fingered exoskeleton haptic device using passive force feedback for dexterous teleoperation , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Sajad Shirali-Shahreza,et al.  Examining the usage of feedback vibration in Nintendo DS handheld game console , 2009, 2009 11th International Conference on Advanced Communication Technology.

[9]  Nikolaos G. Tsagarakis,et al.  HEXOTRAC: A highly under-actuated hand exoskeleton for finger tracking and force feedback , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Sriram Subramanian,et al.  Mid-Air Haptics and Displays: Systems for Un-instrumented Mid-air Interactions , 2016, CHI Extended Abstracts.

[11]  Lorna M. Brown,et al.  Tactile feedback for mobile interactions , 2007, CHI.

[12]  Seung-Chan Kim,et al.  Tactile rendering of 3D features on touch surfaces , 2013, UIST.

[13]  Mike Sinclair,et al.  TouchMover 2.0 - 3D touchscreen with force feedback and haptic texture , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[14]  Hiroshi Ishida,et al.  Smelling Screen: Development and Evaluation of an Olfactory Display System for Presenting a Virtual Odor Source , 2013, IEEE Transactions on Visualization and Computer Graphics.

[15]  Huihui Wang,et al.  Haptics using a smart material for eyes-free interaction in personal devices , 2014, Smart Structures.

[16]  Robert J. K. Jacob,et al.  The VR Scooter: Wind and Tactile Feedback Improve User Performance , 2006, 3D User Interfaces (3DUI'06).

[17]  Mark J.P. Wolf,et al.  The video game explosion : a history from PONG to PlayStation and beyond , 2008 .

[18]  T. Jung,et al.  The determinants of recommendations to use augmented reality technologies: the case of a Korean theme park. , 2015 .

[19]  A. Sloman,et al.  A microcontroller-based driver to stabilize the temperature of an optical stage to within 1 mK in the range , using a Peltier heat pump and a thermistor sensor , 1996 .

[20]  Sriram Subramanian,et al.  Adding haptic feedback to mobile tv , 2011, CHI Extended Abstracts.

[21]  Tomohiro Tanikawa,et al.  A Study of Multi-modal Display System with Visual Feedback , 2008, 2008 Second International Symposium on Universal Communication.

[22]  SubramanianSriram,et al.  Rendering volumetric haptic shapes in mid-air using ultrasound , 2014 .

[23]  Desney S. Tan,et al.  AirWave: non-contact haptic feedback using air vortex rings , 2013, UbiComp.

[24]  Ali Israr,et al.  AIREAL: interactive tactile experiences in free air , 2013, ACM Trans. Graph..

[25]  Y. Yanagida,et al.  A survey of olfactory displays: Making and delivering scents , 2012, 2012 IEEE Sensors.

[26]  Tomohiro Tanikawa,et al.  Wearable Olfactory Display: Using Odor in Outdoor Environment , 2006, IEEE Virtual Reality Conference (VR 2006).

[27]  Sriram Subramanian,et al.  Rendering volumetric haptic shapes in mid-air using ultrasound , 2014, ACM Trans. Graph..

[28]  Hideki Hashimoto,et al.  20 DOF Five Fingered Glove Type Haptic Interface - Sensor Glove II - , 1997, J. Robotics Mechatronics.

[29]  Chris Harrison,et al.  Providing dynamically changeable physical buttons on a visual display , 2009, CHI.

[30]  Gabriel-Miro Muntean,et al.  Perceived Synchronization of Mulsemedia Services , 2015, IEEE Transactions on Multimedia.

[31]  A. Kheddar,et al.  Thermal feedback model for virtual reality , 2003, MHS2003. Proceedings of 2003 International Symposium on Micromechatronics and Human Science (IEEE Cat. No.03TH8717).

[32]  Takuji Narumi,et al.  Pseudo-gustatory display system based on cross-modal integration of vision, olfaction and gustation , 2011, 2011 IEEE Virtual Reality Conference.

[33]  Gerard Jounghyun Kim,et al.  Design and evaluation of a wind display for virtual reality , 2004, VRST '04.

[34]  Renata Rybarova,et al.  Application of immersive technologies for education: State of the art , 2015, 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL).

[35]  Ellen Yi-Luen Do,et al.  Digital Lollipop , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[36]  Kouta Minamizawa,et al.  Haptic transmission system to recognize differences in surface textures of objects for telexistence , 2013, 2013 IEEE Virtual Reality (VR).

[37]  Ali Israr,et al.  TeslaTouch: electrovibration for touch surfaces , 2010, UIST.

[38]  Allison M. Okamura,et al.  Closed-loop shape control of a Haptic Jamming deformable surface , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).