Gesture recognition system for real-time mobile robot control based on inertial sensors and motion strings

Abstract Navigating and controlling a mobile robot in an indoor or outdoor environment by using a range of body-worn sensors is becoming an increasingly interesting research area in the robotics community. In such scenarios, hand gestures offer some unique capabilities for human–robot interaction inherent to nonverbal communication with features and application scenarios not possible with the currently predominant vision-based systems. Therefore, in this paper, we propose and develop an effective inertial-sensor-based system, worn by the user, along with a microprocessor and wireless module for communication with the robot at distances of up to 250 m. Possible features describing hand-gesture dynamics are introduced and their feasibility is demonstrated in an off-line scenario by using several classification methods (e.g., random forests and artificial neural networks). Refined motion features are then used in K-means unsupervised clustering for motion primitive extraction, which forms the motion strings used for real-time classification. The system demonstrated an F 1 score of 90 . 05 % with the possibility of gesture spotting and null class classification (e.g., undefined gestures were discarded from the analysis). Finally, to demonstrate the feasibility of the proposed algorithm, it was implemented in an Arduino-based 8 -bit ATmega2560 microcontroller for control of a mobile, tracked robot platform.