Implementation of real-time motion and force capturing system for tele-manipulation based on sEMG signals and IMU motion data

In this paper, we present a real-time motion and force capturing system for tele-operated robotic manipulation that combines surface-electromyogram (sEMG) pattern recognition with an inertia measurement unit(IMU) for motion calculation. The purpose of this system is to deliver the human motion and intended force to a remote robotic manipulator and to realize multi-fingered activities-of-daily-living (ADL) tasks that require motion and force commands simultaneously and instantaneously. The proposed system combines two different sensors: (i) the IMU captures arm motion, (ii) and the sEMG detects the hand motion and force. We propose an algorithm to calculate the human arm motion using IMU sensors and a pattern recognition algorithm for a multi-grasp myoelectric control method that uses sEMG signals to determine the hand postures and grasping force information. In order to validate the proposed motion and force capturing system, we used the in-house developed robotic arm, K-Arm, which has seven degrees-of-freedom (three for shoulder, one for elbow, and three for wrist), and a sixteen degrees-of-freedom robotic hand. Transmission Control Protocol Internet Protocol (TCP/IP)-based network communication was implemented for total system integration. The experimental results verified the effectiveness of the proposed method, although some open problems encountered.