Signer independent isolated Italian sign recognition based on hidden Markov models

Sign languages represent the most natural way to communicate for deaf and hard of hearing. However, there are often barriers between people using this kind of languages and hearing people, typically oriented to express themselves by means of oral languages. To facilitate the social inclusiveness in everyday life for deaf minorities, technology can play an important role. Indeed many attempts have been recently made by the scientific community to develop automatic translation tools. Unfortunately, not many solutions are actually available for the Italian Sign Language (Lingua Italiana dei Segni—LIS) case study, specially for what concerns the recognition task. In this paper, the authors want to face such a lack, in particular addressing the signer-independent case study, i.e., when the signers in the testing set are to included in the training set. From this perspective, the proposed algorithm represents the first real attempt in the LIS case. The automatic recognizer is based on Hidden Markov Models (HMMs) and video features have been extracted using the OpenCV open source library. The effectiveness of the HMM system is validated by a comparative evaluation with Support Vector Machine approach. The video material used to train the recognizer and testing its performance consists in a database that the authors have deliberately created by involving 10 signers and 147 isolated-sign videos for each signer. The database is publicly available. Computer simulations have shown the effectiveness of the adopted methodology, with recognition accuracies comparable to those obtained by the automatic tools developed for other sign languages.