Enhanced dynamic programming approach for subunit modelling to handle segmentation and recognition ambiguities in sign language

Abstract Sign language serves as a primary means of communication among the deaf impaired community. The major challenges faced by the Sign Language Recognition (SLR) system are recognizing signs from large vocabularies in continuous video sequences. In this research paper, a novel subunit sign modelling framework is proposed for vision-based SLR which aims in solving the major issues in SLR systems. The problem of hand segmentation ambiguities and segregating epentheses movements between two adjacent signs in continuous video sequences are addressed. A novel subunit sign modelling framework is presented and illustrated to embark upon these problems while considering large-vocabularies. This framework is developed using a novel methodology of Enhanced Dynamic Programming (EDP) approach in subunit sign modelling. This EDP framework works with a combination of dynamic time warping and spatiotemporal clustering techniques. Since, sign language consists of both spatial and temporal feature vectors, dynamic time warping is used as a distance measure to compute the distance between two adjacent signs in sign trajectories. This distance is used as a temporal feature vector during the clustering of spatial feature vectors using Minimum Entropy Clustering (MEC). This process is done recursively to cluster all the epentheses movements dynamically without using any explicit or implicit modelling. Experimental results have confirmed that the computation cost of the proposed system is less because the epenthesis movements are eliminated before classification and the gesture base space utilized by the sign gestures is very low because the proposed system does not require any modelling to handle epenthesis movements. The results obtained from the proposed subunit sign modelling framework is compared with other existing models in order to prove that the proposed system is best among the existing systems.