Effective hand segmentation and gesture recognition for browsing web pages on a large screen

Modern digital family technology enables people surf the Internet and watch videos via a large screen. This paper proposes an effective scheme for using hand gestures rather than the common remote controllers to browse the web pages on a large TV screen. The proposed scheme models four gesture modes: mouse mode, scroll mode, zoom mode and input mode to help the user browse Web pages naturally and comfortably. Then we combine RGB, depth, motion information and face detection to achieve accurate and real-time hand segmentation and gesture recognition for enabling the four gesture modes. The experiments show the proposed scheme works well in various illumination environments and complicated backgrounds with multiple moving humans. The recognition accuracy of hand shapes in the proposed scheme arrives at 98.50%, and the successful rate for visual digits input reaches 89.00%. Furthermore, the frame rate of the hand-gesture detection and recognition is about 18 fps. Thus the scheme is accurate, real-time and natural.

[1]  Shijie Chen,et al.  Gesture Recognition Techniques in Handwriting Recognition Application , 2010, 2010 12th International Conference on Frontiers in Handwriting Recognition.

[2]  Luc Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, WACV.

[3]  Andrew Zisserman,et al.  Hand detection using multiple proposals , 2011, BMVC.

[4]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[5]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[6]  Yong Wang,et al.  Using human body gestures as inputs for gaming via depth analysis , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[7]  Chandrika Kamath,et al.  Robust techniques for background subtraction in urban traffic video , 2004, IS&T/SPIE Electronic Imaging.

[8]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[9]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Jovan Popović,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH 2009.

[11]  Albrecht Schmidt,et al.  Requirements and design space for interactive public displays , 2010, ACM Multimedia.

[12]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[13]  Discriminant Subspace,et al.  PATTERN ANALYSIS AND MACHINE INTELLIGENCE A publication of the IEEE Computer Society , 2007 .

[14]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[15]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[16]  Yoshua Bengio,et al.  Boosting Neural Networks , 2000, Neural Computation.