Thai Sign Language Recognition with Leap Motion

Sign language recognition proves to be an effective mean for the deaf and those with difficulty of hearing to communicate with the rest, especially among the English speaking communities. Unfortunately, Thai Sign Language (TSL) recognition presents a much higher level of difficulty due to the nature of the language. This paper proposes a framework using Leap Motion with a Deep Learning algorithm to recognition TSL words. A preliminary evaluation confirmed the ability of Leap Motion sensor readings in classifying gestures in TSL. A data augmentation method based on adjustment of gesture speeds was deployed and shown to be beneficial to model training. We evaluated several setups to measure the recognition accuracy of our framework. The best setup achieves 85% accuracy for known speakers of TSL that is in the training data and 77% for the unknown speaker case.