A Real-Time Hand Pose Recognition Method with Hidden Finger Prediction

ABSTRACT In this paper, we present a real-time hand pose recognition method to provide an intuitive user interface through hand poses or movements without a keyboard and a mouse. For this, the areas of right and left hands are segmented from the depth camera image, and noise removal is performed. Then, the rotation angle and the centroid point of each hand area are calculated. Subsequently, a circle is expanded at regular intervals from a centroid point of the hand to detect joint points and end points of the finger by obtaining the midway points of the hand boundary crossing. Lastly, the matching between the hand information calculated previously and the hand model of previous frame is performed, and the hand model is recognized to update the hand model for the next frame. This method enables users to predict the hidden fingers through the hand model information of the previous frame using temporal coherence in consecutive frames. As a result of the experiment on various hand poses with the hidden fingers using both hands, the accuracy showed over 95% and the performance indicated over 32 fps. The proposed method can be used as a contactless input interface in presentation, advertisement, education, and game applications. Keywords : Hand Pose Recognition, Hand-based User Interface, Hand Model

[1]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Li Bai,et al.  Virtual Touch Screen for Mixed Reality , 2004, ECCV Workshop on HCI.

[3]  Shashank Sonkar,et al.  Hand Gesture Recognition Using Microsoft ’ s Kinect , 2012 .

[4]  Guizhong Liu,et al.  Hand tracking based on the combination of 2D and 3D model in gaze-directed video , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[5]  Michael G. Strintzis,et al.  Real-time hand posture recognition using range data , 2008, Image Vis. Comput..

[6]  Björn Stenger,et al.  Hand Pose Estimation Using Hierarchical Detection , 2004, ECCV Workshop on HCI.

[7]  Kanad K. Biswas,et al.  Gesture recognition using Microsoft Kinect® , 2011, The 5th International Conference on Automation, Robotics and Applications.

[8]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Liu Yun,et al.  A Hand Gesture Recognition Method Based on Multi-Feature Fusion and Template Matching , 2012 .

[10]  Jerald Siby,et al.  Hand Gesture Recognition , 2022, SSRN Electronic Journal.

[11]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[12]  Min-Young Na,et al.  An Illumination and Background-Robust Hand Image Segmentation Method Based on the Dynamic Threshold Values , 2011 .

[13]  Ji-Sang Yoo,et al.  Histogram Based Hand Recognition System for Augmented Reality , 2011 .

[14]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  David Geer.  Will Gesture-Recognition Technology Point the Way? , 2004, Computer.