Anatomically Designed Triboelectric Wristbands with Adaptive Accelerated Learning for Human–Machine Interfaces

Recent advances in flexible wearable devices have boosted the remarkable development of devices for human–machine interfaces, which are of great value to emerging cybernetics, robotics, and Metaverse systems. However, the effectiveness of existing approaches is limited by the quality of sensor data and classification models with high computational costs. Here, a novel gesture recognition system with triboelectric smart wristbands and an adaptive accelerated learning (AAL) model is proposed. The sensor array is well deployed according to the wrist anatomy and retrieves hand motions from a distance, exhibiting highly sensitive and high‐quality sensing capabilities beyond existing methods. Importantly, the anatomical design leads to the close correspondence between the actions of dominant muscle/tendon groups and gestures, and the resulting distinctive features in sensor signals are very valuable for differentiating gestures with data from 7 sensors. The AAL model realizes a 97.56% identification accuracy in training 21 classes with only one‐third operands of the original neural network. The applications of the system are further exploited in real‐time somatosensory teleoperations with a low latency of <1 s, revealing a new possibility for endowing cyber‐human interactions with disruptive innovation and immersive experience.

[1]  Jiajie Guo,et al.  Wearable triboelectric devices for haptic perception and VR/AR applications , 2022, Nano Energy.

[2]  Dongrui Wu,et al.  Adhesive and Hydrophobic Bilayer Hydrogel Enabled On‐Skin Biosensors for High‐Fidelity Classification of Human Emotion , 2022, Advanced Functional Materials.

[3]  B. Liu,et al.  Decoding lip language using triboelectric sensors with deep learning , 2022, Nature Communications.

[4]  Sheng Xu,et al.  Soft wearable devices for deep-tissue sensing , 2022, Nature Reviews Materials.

[5]  Zhoupin Yin,et al.  Flexible Mechanical Metamaterials Enabled Electronic Skin for Real‐Time Detection of Unstable Grasping in Robotic Manipulation , 2022, Advanced Functional Materials.

[6]  S. Bensmaia,et al.  The neural mechanisms of manual dexterity , 2021, Nature Reviews Neuroscience.

[7]  Zuankai Wang,et al.  Achieving ultrahigh instantaneous power density of 10 MW/m2 by leveraging the opposite-charge-enhanced transistor-like triboelectric nanogenerator (OCT-TENG) , 2021, Nature Communications.

[8]  Chengkuo Lee,et al.  AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove , 2021, Nature Communications.

[9]  J. Rogers,et al.  Functional Materials and Devices for XR (VR/AR/MR) Applications , 2021, Advanced Functional Materials.

[10]  Chengkuo Lee,et al.  Low cost exoskeleton manipulator using bidirectional triboelectric sensors enhanced multiple degree of freedom sensory system , 2021, Nature Communications.

[11]  Zhong Lin Wang,et al.  Self-powered electro-tactile system for virtual tactile experiences , 2021, Science Advances.

[12]  Dong Chan Kim,et al.  Unconventional Image‐Sensing and Light‐Emitting Devices for Extended Reality , 2021, Advanced Functional Materials.

[13]  Shaoyu Liu,et al.  Materials, Devices, and Systems of On‐Skin Electrodes for Electrophysiological Monitoring and Human–Machine Interfaces , 2020, Advanced science.

[14]  C. Majidi,et al.  Wearable Soft Technologies for Haptic Sensing and Feedback , 2020, Advanced Functional Materials.

[15]  J. Rabaey,et al.  A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition , 2020, Nature Electronics.

[16]  Erik G. Larsson,et al.  Towards 6G wireless communication networks: vision, enabling technologies, and new paradigm shifts , 2020, Science China Information Sciences.

[17]  Conor J Walsh,et al.  Ultra-sensitive and resilient compliant strain gauges for soft machines , 2020, Nature.

[18]  Chengkuo Lee,et al.  Triboelectric nanogenerator sensors for soft robotics aiming at digital twin applications , 2020, Nature Communications.

[19]  Jeongmoon J. Choi,et al.  All-printed nanomembrane wireless bioelectronics using a biocompatible solderable graphene for multimodal human-machine interfaces , 2020, Nature Communications.

[20]  Chenchen Sun,et al.  Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays , 2020, Nature Electronics.

[21]  Chengkuo Lee,et al.  Machine Learning Glove Using Self‐Powered Conductive Superhydrophobic Triboelectric Textile for Gesture Recognition in VR/AR Applications , 2020, Advanced science.

[22]  Zheng Yan,et al.  Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors , 2020 .

[23]  Xiaodong Chen,et al.  Locally coupled electromechanical interfaces based on cytoadhesion-inspired hybrids to identify muscular excitation-contraction signatures , 2020, Nature Communications.

[24]  Qiongfeng Shi,et al.  Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications , 2020, Science Advances.

[25]  S. Ko,et al.  A deep-learned skin sensor decoding the epicentral human motions , 2020, Nature Communications.

[26]  Bin Gao,et al.  Fully hardware-implemented memristor convolutional neural network , 2020, Nature.

[27]  Ganesh R. Naik,et al.  A Piezoresistive Array Armband With Reduced Number of Sensors for Hand Gesture Recognition , 2020, Frontiers in Neurorobotics.

[28]  Zhaoping Li,et al.  A laser-engraved wearable sensor for sensitive detection of uric acid and tyrosine in sweat , 2019, Nature Biotechnology.

[29]  Toshio Tsuji,et al.  A myoelectric prosthetic hand with muscle synergy–based motion determination and impedance model–based biomimetic control , 2019, Science Robotics.

[30]  Zhong Lin Wang,et al.  A bionic stretchable nanogenerator for underwater sensing and energy harvesting , 2019, Nature Communications.

[31]  Jayoung Kim,et al.  Wearable biosensors for healthcare monitoring , 2019, Nature Biotechnology.

[32]  P. Corke,et al.  On the choice of grasp type and location when handing over an object , 2019, Science Robotics.

[33]  Caofeng Pan,et al.  Self‐Powered Tactile Sensor Array Systems Based on the Triboelectric Effect , 2018, Advanced Functional Materials.

[34]  Oussama Khatib,et al.  A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics , 2018, Science Robotics.

[35]  Kaushik Parida,et al.  Skin-touch-actuated textile-based triboelectric nanogenerator with black phosphorus for durable biomechanical energy harvesting , 2018, Nature Communications.

[36]  Zhong Lin Wang,et al.  Eye motion triggered self-powered mechnosensational communication system using triboelectric nanogenerator , 2017, Science Advances.

[37]  Jae Won Lee,et al.  Boosted output performance of triboelectric nanogenerator via electric double layer effect , 2016, Nature Communications.

[38]  Sam Emaminejad,et al.  Fully integrated wearable sensor arrays for multiplexed in situ perspiration analysis , 2016, Nature.

[39]  J. Tour,et al.  Laser-induced porous graphene films from commercial polymers , 2014, Nature Communications.

[40]  Yonggang Huang,et al.  Conformable amplified lead zirconate titanate sensors with enhanced piezoelectric response for cutaneous pressure monitoring , 2014, Nature Communications.