3D Gesture classification with linear acceleration and angular velocity sensing devices for video games

Abstract We present the results of two experiments that explore various aspects of 3D gesture recognition using linear acceleration and angular velocity data. We examine relationships between variables affecting recognition accuracy, including size of gesture set, amount of training data, choice of classifier, and training configuration (user dependent/independent). Using a set of 25 gestures, we first compare the performance of four machine learning algorithms (AdaBoost, SVM, Bayes and Decision Trees) with existing results (Linear Classifier). Next, we investigate how results in existing literature apply to an application-oriented setting. We created a new 3D gesture database comprising 17,890 samples, containing examples of gestures performed in two different settings (a simple data collection setting vs a video game). We then compared the performance of all five classifiers on this new 3D gesture database. Our results indicate that the Linear Classifier can recognize up to 25 gestures at over 99% accuracy when trained in a user dependent configuration. However, in the video game setting, factors such as in-game stress and the ability to recall gestures cause a drop in recognition accuracy to 79%. We present a discussion of possible strategies to improve recognition accuracy in realistic settings by using a combination of recognition algorithms.