A Liquid Metal Based Multimodal Sensor and Haptic Feedback Device for Thermal and Tactile Sensation Generation in Virtual Reality

Virtual reality (VR) has been widely used for training, gaming, and entertainment, and the value of VR is continually increasing as a contact‐free technology. For an immersive VR experience, measuring finger movements and providing appropriate feedback to the hand are as important as visual information, given the necessity of the hands for activities in daily life. Thus, a hand‐worn VR device with motion sensors and haptic feedback is desirable. In this paper, a multimodal sensing and feedback glove is developed with soft, stretchable, lightweight, and compact sensor and heater sheets manufactured by direct ink writing (DIW) of liquid metal, eutectic gallium‐indium (eGaIn). In the sensor sheet, ten sensors and three vibrators are embedded to measure finger movements and provide vibro‐haptic feedback. The other heater sheet provides thermo‐haptic sensation in accurate and rapid manner via model‐based feedback control even under stretched conditions. The multimodal sensing and feedback glove allows users to feel the contact status and discriminate materials with different temperature. Performance of the proposed multimodal glove is verified under VR environments including touching and pushing two blocks of different materials and grabbing a heated metal ball submerged in hot water.

[1]  C. Pehoski,et al.  Hand Function in the Child: Foundations for Remediation , 1994 .

[2]  Cynthia Breazeal,et al.  TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning , 2007, IEEE Transactions on Robotics.

[3]  Robert V Kenyon,et al.  A Pneumatic Glove and Immersive Virtual Reality Environment for Hand Rehabilitative Training After Stroke , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Xing Liang,et al.  Biomechanical Properties of In Vivo Human Skin From Dynamic Optical Coherence Elastography , 2010, IEEE Transactions on Biomedical Engineering.

[5]  M. Nowicki,et al.  FOR LEAP MOTION CONTROLLER , 2014 .

[6]  Yi Cui,et al.  Personal thermal management by metallic nanowire-coated textile. , 2015, Nano letters.

[7]  Yong Wei,et al.  Highly Stable and Sensitive Paper-Based Bending Sensor Using Silver Nanowires/Layered Double Hydroxides Hybrids. , 2015, ACS applied materials & interfaces.

[8]  Q. Wang,et al.  Fast Fabrication of Flexible Functional Circuits Based on Liquid Metal Dual‐Trans Printing , 2015, Advanced materials.

[9]  Ishan D. Joshipura,et al.  Methods to pattern liquid metals , 2015 .

[10]  Pinhas Ben-Tzvi,et al.  RML Glove—An Exoskeleton Glove Mechanism With Haptics Feedback , 2015, IEEE/ASME Transactions on Mechatronics.

[11]  D. Sameoto,et al.  Fabrication methods and applications of microstructured gallium based liquid metal alloys , 2016 .

[12]  B. Hanrahan,et al.  Thermotherapy Platform Based on a Highly Stretchable Wireless Heater , 2016 .

[13]  M. Dickey.  Stretchable and Soft Electronics using Liquid Metals , 2017, Advanced materials.

[14]  Soo Kyun Kim,et al.  Erratum to: Development of a low-cost wearable sensing glove with multiple inertial sensors and a light and fast orientation estimation algorithm , 2017, The Journal of Supercomputing.

[15]  Atulya K. Nagar,et al.  Wearable Vibrotactile Haptic Device for Stiffness Discrimination during Virtual Interactions , 2017, Front. Robot. AI.

[16]  George-Christopher Vosniakos,et al.  Design of a virtual reality training system for human–robot collaboration in manufacturing tasks , 2015, International Journal on Interactive Design and Manufacturing (IJIDeM).

[17]  Yuhao Liu,et al.  Lab-on-Skin: A Review of Flexible and Stretchable Electronics for Wearable Health Monitoring. , 2017, ACS nano.

[18]  Haiyan Wu,et al.  Ionic Conductivity of Polyelectrolyte Hydrogels. , 2018, ACS applied materials & interfaces.

[19]  Arnaldo G. Leal-Junior,et al.  Polymer Optical Fiber Sensors in Wearable Devices: Toward Novel Instrumentation Approaches for Gait Assistance Devices , 2018, IEEE Sensors Journal.

[20]  M. Ma,et al.  An innovative virtual reality training tool for orthognathic surgery. , 2018, International journal of oral and maxillofacial surgery.

[21]  Hend Suliman Al-Khalifa,et al.  RabbitRun: An Immersive Virtual Reality Game for Promoting Physical Activities Among People with Low Back Pain , 2018, Technologies.

[22]  J. Bae,et al.  Consistent and Reproducible Direct Ink Writing of Eutectic Gallium-Indium for High-Quality Soft Sensors. , 2018, Soft robotics.

[23]  Doyoon Kim,et al.  Towards sub-microscale liquid metal patterns: Cascade phase change mediated pick-n-place transfer of liquid metals printed and stretched over a flexible substrate , 2018, 2018 IEEE Micro Electro Mechanical Systems (MEMS).

[24]  Yiwei Liu,et al.  Printable Liquid‐Metal@PDMS Stretchable Heater with High Stretchability and Dynamic Stability for Wearable Thermotherapy , 2018, Advanced Materials Technologies.

[25]  Joan Condell,et al.  IMU Sensor-Based Electronic Goniometric Glove for Clinical Finger Movement Analysis , 2018, IEEE Sensors Journal.

[26]  Lianqing Zhu,et al.  Soft optical fiber curvature sensor for finger joint angle proprioception , 2019, Optik.

[27]  Jingyan Dong,et al.  High-performance low-voltage soft electrothermal actuator with directly printed micro-heater , 2019, Sensors and Actuators A: Physical.

[28]  Hong Liu,et al.  High‐Resolution Patterning of Liquid Metal on Hydrogel for Flexible, Stretchable, and Self‐Healing Electronics , 2019, Advanced Electronic Materials.

[29]  Jianzhong Fu,et al.  All‐Printed Flexible and Stretchable Electronics with Pressing or Freezing Activatable Liquid‐Metal–Silicone Inks , 2019, Advanced Functional Materials.

[30]  Wenlong Cheng,et al.  Electronic Skins Based on Liquid Metals , 2019, Proceedings of the IEEE.

[31]  Pooi See Lee,et al.  Progress on triboelectric nanogenerator with stretchability, self-healability and bio-compatibility , 2019, Nano Energy.

[32]  J. Bae,et al.  Direct Wiring of Eutectic Gallium-Indium to a Metal Electrode for Soft Sensor Systems. , 2019, ACS applied materials & interfaces.

[33]  Ben Wang,et al.  Liquid Metal-Based Soft Microfluidics. , 2020, Small.

[34]  Dae-Hyeong Kim,et al.  Materials and Design Strategies of Stretchable Electrodes for Electronic Skin and its Applications , 2019, Proceedings of the IEEE.

[35]  Ping Peng,et al.  Tailoring vertical phase distribution of quasi-two-dimensional perovskite films via surface modification of hole-transporting layer , 2019, Nature Communications.

[36]  J. Y. Kim,et al.  Walking Simulation for VR Game Character Using Remote Sensoring Device Based on AHRS-Motion Recognition , 2019, IEEE Access.

[37]  Hyeon Seok An,et al.  High-resolution, reconfigurable printing of liquid metals with three-dimensional structures , 2019, Science Advances.

[38]  M. Kunitski,et al.  Double-slit photoelectron interference in strong-field ionization of the neon dimer , 2018, Nature Communications.

[39]  Jeongsoo Lee,et al.  WeHAPTIC: A Wearable Haptic interface for Accurate Position Tracking and Interactive force Control , 2020 .

[40]  Zhong Lin Wang,et al.  β-Phase-Preferential blow-spun fabrics for wearable triboelectric nanogenerators and textile interactive interface , 2020 .

[41]  C. Majidi,et al.  Ultrastretchable, Wearable Triboelectric Nanogenerator Based on Sedimented Liquid Metal Elastomer Composite , 2020, Advanced Materials Technologies.

[42]  Zhong Lin Wang,et al.  A universal and arbitrary tactile interactive system based on self-powered optical communication , 2020 .

[43]  Zhong Lin Wang,et al.  Stretchable Energy‐Harvesting Tactile Interactive Interface with Liquid‐Metal‐Nanoparticle‐Based Electrodes , 2020, Advanced Functional Materials.

[44]  Teresa A Kent,et al.  Soft actuators using liquid crystal elastomers with encapsulated liquid metal joule heaters , 2020, Multifunctional Materials.

[45]  S. Ko,et al.  Stretchable Skin‐Like Cooling/Heating Device for Reconstruction of Artificial Thermal Sensation in Virtual Reality , 2020, Advanced Functional Materials.

[46]  Jing Liu,et al.  Liquid metal–enabled cybernetic electronics , 2020 .

[47]  S. Ko,et al.  Highly stretchable and oxidation-resistive Cu nanowire heater for replication of the feeling of heat in a virtual world , 2020 .