A user independent hand gesture recognition system using deep CNN feature fusion and machine learning technique

Abstract Hand gesture recognition is one of the active research areas in the field of human-computer interface due to its flexibility and user friendliness. The gesture recognition technique is used to develop a system that can be used to convey information among disabled people or for controlling a device. Major challenges for the development of an efficient hand gesture recognition technique are illumination variation, nonuniform backgrounds, diversities in the size and shape of a user's hand, and high interclass similarities between hand gesture poses. In this chapter, a user independent static hand gesture recognition technique is analyzed using handcrafted features such as a histogram of oriented gradients (HOG) and a deep convolutional neural network (CNN). The deep features are extracted from fully connected layers of two different well-known pretrained CNNs such as AlexNet and VGG-16. The fusion of feature vectors extracted from different fully connected layers of both the CNNs is proposed for the enhancement of gesture recognition accuracy. The proposed CNN-based feature does not require any hand segmentation or background subtraction technique to segment the hand region from the input image. A support vector machine as a machine learning algorithm is used to classify the gesture poses. A comparison of the HOG feature and the deep CNN feature is presented for the recognition of static hand gesture poses. The proposed deep CNN feature fusion-based hand gesture recognition is robust to illumination variation, nonuniform backgrounds, and interclass similarities. The performance of the proposed technique is evaluated using leave-one-subject-out cross-validation (LOO CV) and holdout CV tests. The extensive analysis is performed on three benchmark static hand gesture datasets with uniform and nonuniform backgrounds on both the CV tests. A significant improvement of user independent static hand gesture recognition performance using the LOO CV test is found using the proposed technique. The experimental results show that the proposed technique is superior as compared to individual deep CNN features and state-of-the-art techniques. A real-time application of the gesture recognition system is developed and tested using the proposed technique.