Gestures with speech for hand-impaired persons

Mid-air hand-gestural interaction generally causes a fatigue due to implementations that require the user to hold their arm out during this interaction. Recent research has discovered a new approach to reduce fatigue related to gestural interaction, by allowing users to rest their elbow on a surface, and calibrate their interaction space from this rested position[1]. Additionally, this approach reduced stress on the hand and wrist compared to the mouse, by shifting much of the load to the forearm and shoulder muscles. In this paper we evaluated gesture and speech multimodal interaction as a form of assistive interaction for those with hand impairments. Two participants with hand impairments were recruited to perform the evaluation. We collected qualitative and quantitative data, which showed promising results in using this method for assistive interaction.