ICANDO: Low cost multimodal interface for hand disabled people

The article presents the multimodal user interface ICANDO (Intellectual Computer AssistaNt for Disabled Operators) that was awarded with the first prize of the Loco Mummy Software Contest in 2006. The interface is intended mainly for assistance to the persons without hands or with disabilities of their hands or arms but could be useful for ordinary users at hands-free contactless human-computer interaction too. It combines the module for automatic recognition of voice commands in English, French and Russian as well as the head tracking module in one multimodal interface. ICANDO interface was applied for hands-free work with Graphical User Interface of a personal computer in such tasks as Internet communication and work with graphical and text documents. The article describes the aim and the architecture of the interface, the methods for speech recognition and head tracking, information fusion and synchronization of the multimodal streams. The presented results of testing and exploitation of ICANDO user interface have confirmed high accuracy and robustness of the interface for contactless operation with a computer. The comparison of multimodal and standard ways of interaction has discovered that the first one is slower by a factor of 1.9 that is quite well for hands-free interaction between a computer and an impaired person.