Give Me a Sign : A Person Independent Interactive Sign Dictionary

AbstractThis paper presents a method to perform person independent sign recog-nition. This is achieved by implementing generalising features based onsign linguistics. These are combined using two methods. The first is tradi-tional Markov models, which are shown to lack the required generalisation.The second is a discriminative approach called Sequential Pattern Boosting,which combines feature selection with learning. The resulting system is in-troduced as a dictionary application, allowing signers to query by perform-ing a sign in front of a Kinect TM . Two data sets are used and results shownfor both, with the query-return rate reaching 99.9% on a 20 sign multi-userdataset and 85.1% on a more challenging and realistic subject independent,40 sign test set. 1 Introduction While image indexes into search engines are becoming common place, the abilityto search using an action or gesture is still an open research question. For sign lan-guage users, this makes looking up new signs in a dictionary a non-trivial task. Allexisting sign language dictionaries are complex to navigate due to the lack of uni-versal indexing feature (like the alphabet in written language). This work attemptsto address this by proposing an interactive sign dictionary. The proposed dictionarycan be queried by enacting the sign, live, in front of a Microsoft Kinect