A real-time gesture prediction system using neural networks and multimodal fusion based on data glove

Unlike static gesture recognition, a novel real-time gesture prediction system in this study can judge the intention of hand motion and predict the exact final gesture before the end of hand movement. Flex sensors are used to measure comprehensive motion data of data glove, which are positioned based on the biological muscle distribution characteristics of the hand. Position, velocity and acceleration information are extracted from raw data of data glove, while the adjacent finger-coupling features are also obtained by processing the position and velocity information. After data processing such as windowing and filtering, accuracy and effectiveness experiments are conducted to obtain the ideal features based on multimodal fusion. A combination of neural network and multiclass support vector machine (SVM) algorithms are used as prediction model. Neural network experiments are designed in which prediction time and accuracy are used as the optimization index to select the combination structure of the prediction model.