Gesture Aided Speech for Deaf and Mute

Speech impaired people make use of sign language to communicate along with normal people. Common people also face difficulties to understand the gesture language. In order to minimize these real time issues, an attempt has been made to develop a system which consists of Flex sensors attached with the Data gloves along all fingers. These will be wearable gloves converts hand gestures into an audio output to interpret the expression. This reduces the communication gap that exists between mute and ordinary people. The proposed system plays corresponding recorded voice on an android phone connected to the hardware system via Bluetooth Module as an interpretation of the gesture. It is achieved by integrating flex sensor and 8051 microcontroller. The phone will use a text to speech converter to produce the output of corresponding gesture. This system offers high reliability and fast response as microcontroller works at a speed of 12 MHz. Processing time of microcontroller is much less than the time taken for a human to change from one gesture to another, the system works efficiently in real time scenario hence making the system faster.

[1]  Ibrahim Patel,et al.  AUTOMATED SPEECH SYNTHESIZER AND CONVERTER IN CUE SYMBOL GENERATION FOR HEARING IMPARDS , 2009 .

[2]  Matthew Stone,et al.  Modeling the Interaction between Speech and Gesture. , 1994 .

[3]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[4]  Abd Aziz Mohd Azri,et al.  The development of a low cost cata glove by using flexible bend sensor for resistive interfaces , 2012 .

[5]  Geoffrey E. Hinton,et al.  Glove-TalkII: Mapping Hand Gestures to Speech Using Neural Networks , 1994, NIPS.

[6]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[7]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  M. Sornalatha,et al.  Hand gesture recognition and voice conversion system for dumb people , 2014 .

[9]  S Vigneshwaran,et al.  Hand Gesture Recognition and Voice Conversion System for Dump People , 2019 .

[10]  AMAZING DIGITAL GLOVES,et al.  The Amazing Digital Gloves That Give Voice to the Voiceless , 2013 .

[11]  Balaji,et al.  Give Voice to the Voiceless Using Microcontroller and Digital Gloves , 2014 .

[12]  乔宇 Gesture Design of Hand-to-Speech Converter derived from Speech-to-Hand Converter based on Probabilistic Integration Model , 2011 .

[13]  Dennis McLeod,et al.  An Approach to Glove-Based Gesture Recognition , 2009, HCI.

[14]  Priyanka R. Potdar,et al.  Innovative Approach for Gesture to Voice Conversion: Review , 2014 .