Dynamic audiotactile feedback in gesture interaction

Proper feedback is one of the challenges in gesture interaction. Providing continuous feedback during the execution of the gesture increases the feeling of control and it can help user to perform the task more efficiently. In this paper we introduce an experimental handheld sensor-actuator device that responds dynamically to user's motion. With the device we explored the potential of continuous audiotactile feedback in closed-loop gesture interaction, designed simple synthesis methods for feedback, and tested the user perception. We designed four simple textures that respond to overall angular velocity of the device, all with different velocity responses. The system enabled us to examine how well subjects can distinguish the textures on the fly. Our preliminary findings show that audio modality dominates the perception. Tactile feedback worked quite well alone but the modalities together didn't lead to any better performance than audio alone.