AN EYES-FREE USER INTERFACE CONTROLLED BY FINGER SNAPS

A novel way of controlling a simple user interface based on detecting and localizing finger snaps of the user is presented. The analysis method uses binaural signals recorded from the ears of the user. Transient sounds are first detected from a continuous audio stream, followed by cross-correlation based localization and simple band-energy ratio based classification. The azimuth plane around the user is divided into three sectors, each of which corresponds to one of the three “buttons” in the interface. As an example, the interface is applied for controlling the playlist of an MP3 player. The algorithm performance was evaluated using a real-world recording. While the algorithm looks promising, more research is needed before it is ready for commercial applications.

[1]  Sampo Vesa,et al.  Automatic estimation of reverberation time from binaural signals , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[2]  Tommi Ilmonen.  Mustajuuri - An application and toolkit for interactive audio processing , 2001 .

[3]  M. Tikander,et al.  Binaural positioning system for wearable augmented reality audio , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[4]  C. Faller,et al.  Source localization in complex listening situations: selection of binaural cues based on interaural coherence. , 2004, The Journal of the Acoustical Society of America.

[5]  Tapio Lokki,et al.  Augmented reality audio for mobile and wearable appliances , 2004 .

[6]  Thomas Wittkop,et al.  Two-channel noise reduction algorithms motivated by models of binaural interaction , 2001 .

[7]  Tapio Lokki,et al.  Techniques and Applications of Wearable Augmented Reality Audio , 2003 .

[8]  Stephen A. Brewster,et al.  A Study on Gestural Interaction with a 3D Audio Display , 2004, Mobile HCI.

[9]  Stephen A. Brewster,et al.  Multimodal 'eyes-free' interaction techniques for wearable devices , 2003, CHI '03.

[10]  Vesa T. Peltonen,et al.  Computational auditory scene recognition , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Harald Viste,et al.  On the Use of Spatial Cues to Improve Binaural Source Separation , 2003 .