Beatbox music phone: gesture-based interactive mobile phone using a tri-axis accelerometer

In this paper, we propose gesture-based interaction methods using a tri-axis accelerometer for mobile devices. The algorithms for the gesture recognition and the shaking detection are suggested for gesture interaction. They have been implemented in the commercialized mobile phone (model name: SCH-S310). The mobile phone recognizes digits from 1 to 9 and five symbols written in the air. It detects the shaking motion to initiate commands for the gesture interactive games and musical instrument applications. We used the signals from a tri-axis accelerometer to recognize the gestures directly without trajectory estimation. In the experimental study, we achieved 97.01% of average recognition rate for a set of eleven gestures. The rate is cross validated from a whole data set of 3082 gestures from 100 users. The performance of the shaking detection methods shows that gesture-based interaction can be used as input methods of games and musical instrument applications for entertainment

[1]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Sargur N. Srihari,et al.  On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Tsukasa Ogasawara,et al.  Hand pose estimation for vision-based human interface , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[4]  Kazunori Itoh,et al.  Handwritten Pattern Reproduction Using 3D Inertial Measurement of Handwriting Movement , 2002 .

[5]  Tamás Szirányi,et al.  Supervised training based hand gesture recognition system , 2002, Object recognition supported by user interaction for service robots.

[6]  Kyoung-Ho Kang,et al.  Self-contained spatial input device for wearable computers , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[7]  Jani Mäntyjärvi,et al.  Enabling fast and effortless customisation in accelerometer based gesture interaction , 2004, MUM '04.

[8]  Jing Yang,et al.  Development of the gyro-free handwriting input device based on inertial navigation system (INS) theory , 2004, SICE 2004 Annual Conference.

[9]  Jing Yang,et al.  Magic wand: a hand-drawn gesture input device in 3-D space with inertial sensors , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[10]  Sung-Jung Cho,et al.  Bayesian network modeling of strokes and their relationships for on-line handwriting recognition , 2004, Pattern Recognit..