Chinese Sign Language Recognition Based on Multiple Sensors Information Detection and Fusion

The efficient fusion of hand gesture information captured by a three-axis accelerometer,a webcam and four surface electromyography sensors is an important research field for improving the performance of sign language recognition system.In this paper,a multi-sensor information detection and fusion method was proposed.Firstly,the amplitude information of myoelectric signal was utilized to extract active segments of hand gestures and divide sign gestures into single-hand type and double-hand type.Then double-hand sign words were further classified into occlusion or non-occlusion class by vision signal.Lastly,decision-level fusion approach with Sugeno fuzzy integral was applied on local matching results of multiple classifiers for improving classification performance.Experimental results for 201 high-frequency sign words from 4 signers obtained the classification accuracy of more than 99%,indicating the effectiveness of the proposed method for Chinese sign language recognition.