Increasing Electrical Muscle Stimulation's Dexterity by means of Back of the Hand Actuation

We propose a technique that allows an unprecedented level of dexterity in electrical muscle stimulation (EMS), i.e., it allows interactive EMS-based devices to flex the user's fingers independently of each other. EMS is a promising technique for force feedback because of its small form factor when compared to mechanical actuators. However, the current EMS approach to flexing the user's fingers (i.e., attaching electrodes to the base of the forearm, where finger muscles anchor) is limited by its inability to flex a target finger's metacarpophalangeal (MCP) joint independently of the other fingers. In other words, current EMS devices cannot flex one finger alone, they always induce unwanted actuation to adjacent fingers. To tackle the lack of dexterity, we propose and validate a new electrode layout that places the electrodes on the back of the hand, where they stimulate the interossei/lumbricals muscles in the palm, which have never received attention with regards to EMS. In our user study, we found that our technique offers four key benefits when compared to existing EMS electrode layouts: our technique (1) flexes all four fingers around the MCP joint more independently; (2) has less unwanted flexion of other joints (such as the proximal interphalangeal joint); (3) is more robust to wrist rotations; and (4) reduces calibration time. Therefore, our EMS technique enables applications for interactive EMS systems that require a level of flexion dexterity not available until now. We demonstrate the improved dexterity with four example applications: three musical instrumental tutorials (piano, drum, and guitar) and a VR application that renders force feedback in individual fingers while manipulating a yo-yo.

[1]  Pedro Lopes,et al.  Adding Force Feedback to Mixed Reality Experiences and Games using Electrical Muscle Stimulation , 2018, CHI.

[2]  Jonna Häkkilä,et al.  EMS Painter: Co-creating Visual Art using Electrical Muscle Stimulation , 2018, TEI.

[3]  Florian Alt,et al.  Supporting interaction in public space with electrical muscle stimulation , 2013, UbiComp.

[4]  Pedro Lopes,et al.  Muscle-propelled force feedback: bringing force feedback to mobile devices , 2013, CHI.

[5]  Stefan Schneegaß,et al.  EMS in HCI: challenges and opportunities in actuating human bodies , 2017, MobileHCI.

[6]  Michael Rohs,et al.  Emotion Actuator: Embodied Emotional Feedback through Electroencephalography and Electrical Muscle Stimulation , 2017, CHI.

[7]  Marc H Schieber,et al.  Human finger independence: limitations due to passive mechanical coupling versus active neuromuscular control. , 2004, Journal of neurophysiology.

[8]  Dingguo Zhang,et al.  Wireless Master-Slave FES Rehabilitation System Using sEMG Control , 2012, ICIRA.

[9]  Yoichi Nagashima,et al.  Bio-Sensing Systems and Bio-Feedback Systems for Interactive Media Arts , 2003, NIME.

[10]  Tomohiko Hayakawa,et al.  Comparison of reaction times in response to electrical and Visual Stimulation using a high-speed camera , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[11]  Takaya Ishimaru,et al.  Virtual bumps display based on electrical muscle stimulation , 2020, 2020 IEEE Haptics Symposium (HAPTICS).

[12]  Yoichi Ochiai,et al.  Building a feedback loop between electrical stimulation and percussion learning , 2018, SIGGRAPH 2018.

[13]  Shubham Sharma,et al.  Star-Force: A Playful Implementation of the Jedi-force , 2017, Tangible and Embedded Interaction.

[14]  Tomohiko Hayakawa,et al.  Trajectory adjustment system for learning based on electrical stimulation , 2017, AH.

[15]  Pedro Lopes,et al.  Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation , 2017, CHI.

[16]  Yasuaki Kakehi,et al.  Paralogue: A Remote Conversation System Using a Hand Avatar which Postures are Controlled with Electrical Muscle Stimulation , 2018, AH.

[17]  Carlo Alberto Avizzano,et al.  Design and Development of a Hand Exoskeleton Robot for Active and Passive Rehabilitation , 2016 .

[18]  Max Pfeiffer,et al.  3D Virtual Hand Selection with EMS and Vibration Feedback , 2015, CHI Extended Abstracts.

[19]  Dieter Schmalstieg,et al.  Using neuromuscular electrical stimulation for pseudo-haptic feedback , 2006, VRST '06.

[20]  Kasper Hornbæk,et al.  Wanding Through Space: Interactive Calibration for Electric Muscle Stimulation , 2018, AH.

[21]  Pedro Lopes,et al.  Action-Dependent Processing of Touch in the Human Parietal Operculum and Posterior Insula. , 2019, Cerebral cortex.

[22]  A. Kralj,et al.  Programmed Six-Channel Electrical Stimulator for Complex Stimulation of Leg Muscles During Walking , 1979, IEEE Transactions on Biomedical Engineering.

[23]  Kai Kunze,et al.  EMS icons: conveying information by analogy to enhance communication through electrical muscle stimulation , 2017, UbiComp/ISWC Adjunct.

[24]  Kenji Suzuki,et al.  Wired muscle: generating faster kinesthetic reaction by inter-personally connecting muscles , 2017, SIGGRAPH Emerging Technologies.

[25]  Farzam Farbiz,et al.  An electrical muscle stimulation haptic feedback for mixed reality tennis game , 2007, SIGGRAPH '07.

[26]  Michael Rohs,et al.  A Wearable Force Feedback Toolkit with Electrical Muscle Stimulation , 2016, CHI Extended Abstracts.

[27]  Akira Ishii,et al.  Stimulated percussions: method to control human for learning music by using electrical muscle stimulation , 2017, AH.

[28]  Xiang Li,et al.  FESleeve: A Functional Electrical Stimulation System with Multi-electrode Array for Finger Motion Control , 2016, ICIRA.

[29]  Pedro Lopes,et al.  Immense Power in a Tiny Package: Wearables Based on Electrical Muscle Stimulation , 2017, IEEE Pervasive Computing.

[30]  Kenji Suzuki,et al.  A wearable stimulation device for sharing and augmenting kinesthetic feedback , 2015, AH.

[31]  Michael Rohs,et al.  Let me grab this: a comparison of EMS and vibration for haptic feedback in free-hand interaction , 2014, AH.

[32]  Xiaoying Lü,et al.  Electrode placement on the forearm for selective stimulation of finger extension/flexion , 2018, PloS one.

[33]  Stefan Schneegaß,et al.  Embodied notifications: implicit notifications through electrical muscle stimulation , 2016, MobileHCI Adjunct.

[34]  M. Popovic,et al.  Multi-field surface electrode for selective electrical stimulation. , 2005, Artificial organs.

[35]  Jun Rekimoto,et al.  PossessedHand: techniques for controlling human hands using electrical muscles stimuli , 2011, CHI.

[36]  Michael Rohs,et al.  Zap++: a 20-channel electrical muscle stimulation system for fine-grained wearable force feedback , 2017, MobileHCI.

[37]  Makoto Oka,et al.  Feedback Control to Target Joints Angle in Middle Finger PIP and MP Joint Using Functional Electrical Stimulation , 2019, HCI.

[38]  Maarten J. IJzerman,et al.  Relation between stimulation characteristics and clinical outcome in studies using electrical stimulation to improve motor control of the upper extremity in stroke. , 2005, Journal of rehabilitation medicine.

[39]  Jeremy R. Cooperstock,et al.  Raising the Heat: Electrical Muscle Stimulation for Simulated Heat Withdrawal Response , 2017, UIST.

[40]  Kenji Suzuki,et al.  bioSync: A Paired Wearable Device for Blending Kinesthetic Experience , 2017, CHI.

[41]  Jun Rekimoto,et al.  Design and Study of a Multi-Channel Electrical Muscle Stimulation Toolkit for Human Augmentation , 2018, AH.

[42]  Albrecht Schmidt,et al.  Creating user interfaces with electrical muscle stimulation , 2017, Interactions.

[43]  Pedro Lopes,et al.  Muscle-plotter: An Interactive System based on Electrical Muscle Stimulation that Produces Spatial Output , 2016, UIST.

[44]  Tomohiko Hayakawa,et al.  Supportive training system for sports skill acquisition based on electrical stimulation , 2017, 2017 IEEE World Haptics Conference (WHC).

[45]  Pedro Lopes,et al.  Affordance++: Allowing Objects to Communicate Dynamic Use , 2015, CHI.

[46]  Dingguo Zhang,et al.  Influence of volitional contraction on muscle response to functional electrical stimulation , 2014, 2014 IEEE 19th International Functional Electrical Stimulation Society Annual Conference (IFESS).

[47]  Nozomu Hoshimiya,et al.  A method of multichannel PID control of two-degree-of-freedom wrist joint movements by functional electrical stimulation , 2003, Systems and Computers in Japan.

[48]  Pedro Lopes,et al.  Proprioceptive Interaction , 2015, CHI.

[49]  Bin Zhang,et al.  Master-Slave Gesture Learning System Based on Functional Electrical Stimulation , 2014, ICIRA.

[50]  Toshiaki Tsuji,et al.  Chattering reduction of functional electrical stimulation with the smith compensator , 2017, IECON 2017 - 43rd Annual Conference of the IEEE Industrial Electronics Society.

[51]  Michael Rohs,et al.  On-skin technologies for muscle sensing and actuation , 2016, UbiComp Adjunct.

[52]  Jun Rekimoto,et al.  Design Guideline for Developing Safe Systems that Apply Electricity to the Human Body , 2018, ACM Trans. Comput. Hum. Interact..

[53]  Toshio Tsuji,et al.  Stiffness Display by Muscle Contraction Via Electric Muscle Stimulation , 2016, IEEE Robotics and Automation Letters.

[54]  Sebastian Boring,et al.  Automatic Calibration of High Density Electric Muscle Stimulation , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[55]  Emi Tamaki,et al.  UnlimitedHand: Input and Output Hand Gestures with Less Calibration Time , 2016, UIST.

[56]  Pedro Lopes,et al.  Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation Without Compromising Agency , 2019, CHI.

[57]  Anna Vallgårda,et al.  FeltRadio: Sensing and Making Sense of Wireless Traffic , 2016, Conference on Designing Interactive Systems.

[58]  Michael Rohs,et al.  Follow the Force: Steering the Index Finger towards Targets using EMS , 2016, CHI Extended Abstracts.

[59]  Hiroyuki Kajimoto,et al.  Electrotactile Display with Real-Time Impedance Feedback Using Pulse Width Modulation , 2012, IEEE Transactions on Haptics.

[60]  Hiroyuki Kajimoto,et al.  Optimal selection of electrodes for muscle electrical stimulation using twitching motion measurement , 2013, AH.