Adaptive Motion-Based Gesture Recognition Interface for Mobile Phones

In this paper, we introduce a new vision based interaction technique for mobile phones. The user operates the interface by simply moving a finger in front of a camera. During these movements the finger is tracked using a method that embeds the Kalman filter and ExpectationMaximization (EM) algorithms. Finger movements are interpreted as gestures using Hidden Markov Models (HMMs). This involves first creating a generic model of the gesture and then utilizing unsupervised Maximum a Posteriori (MAP) adaptation to improve the recognition rate for a specific user. Experiments conducted on a recognition task involving simple control commands clearly demonstrate the performance of our approach.

[1]  Sami Huttunen,et al.  Motion-based finger tracking for user interaction with mobile devices , 2007 .

[2]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  Jianying Hu,et al.  HMM Based On-Line Handwriting Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jean-Marc Odobez,et al.  Multi-modal audio-visual event recognition for football analysis , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[6]  Janne Heikkilä,et al.  Global motion estimation using block matching with uncertainty analysis , 2007, 2007 15th European Signal Processing Conference.

[7]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[8]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[9]  Shumin Zhai,et al.  Camera phone based motion sensing: interaction techniques, applications and performance study , 2006, UIST.

[10]  Tolga K. Çapin,et al.  Mobile Camera-Based User Interaction , 2005, ICCV-HCI.

[11]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[12]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[13]  Lianwen Jin,et al.  A Novel Vision-Based Finger-Writing Character Recognition System , 2007, J. Circuits Syst. Comput..

[14]  Alexander Zelinsky,et al.  Finger Track - A Robust and Real-Time Gesture Interface , 1997, Australian Joint Conference on Artificial Intelligence.

[15]  Ali H. Sayed,et al.  A Robust Finger Tracking Method for Multimodal Wearable Computer Interfacing , 2006, IEEE Transactions on Multimedia.

[16]  Janne Heikkilä,et al.  Vision-based motion estimation for interaction with mobile devices , 2007, Comput. Vis. Image Underst..

[17]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Jari Hannuksela,et al.  A vision based motion interface for mobile phones , 2007 .