Assistive Hand Gesture Glove for Hearing and Speech Impaired

Hand based gestures are words of communication for people impaired of speech and hearing. This results in communication mismatch between such impaired people and a normal one. A deaf and dumb person many times finds difficulty in informing about basic phrases or words representing certain actions like “How are you?”, “Yes”, etc. to a normal human. To tackle this issue, one hand gesture recognition system is presented which uses a sensor and a microcontroller to capture a gesture movement in the form of a signal. This hand gesture recognition system uses 1D Convolutional Neural Network (1D-CNN) which can extract feature directly from the raw temporal signals captured. The resulted word or phrase is communicated to a normal person through the Mobile phone of a disabled person in terms of audio voice and text-based notification. Furthermore, to reduce the latency in predicting output, the trained 1D-CNN model is deployed in Android phone itself rather than running the model on the server. The trained model achieves recognition accuracy of 97.96% on test data consisting of many samples of 10 different patterns.