Real-time hand tracking using integrated optical flow and CAMshift algorithm

Hand gesture interfaces are more convenient, natural, intuitive, and user-friendly form of input for HumanComputer Interaction(HCI). Hand detection and tracking are the most vital stages for any kind of hand gesture based interface and the accuracy of the final gesture recognition algorithm depends substantially on the proper and correct segmentation of hand from incoming video frames in real time. This paper proposes a novel hand tracking algorithm, that combines Continuous Adaptive Mean Shift Algorithm(CAMshift), Shi-Tomasi points, and Lukas Kanade Optical flow to track hand with high accuracy in real time using only a single camera in non-limiting and unrestrained environment. Results obtained reflect that the algorithm can precisely track the hand of an operator in an input video sequence obtained from a web-cam at 30fps.

[1]  Yu-Cheng Fan,et al.  Depth and color-based three dimensional natural user interface , 2016, 2016 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW).

[2]  Gary R. Bradski,et al.  Real time face and object tracking as a component of a perceptual user interface , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[3]  D. Anil Kumar,et al.  Optical Flow Hand Tracking and Active Contour Hand Shape Features for Continuous Sign Language Recognition with Artificial Neural Networks , 2015, 2016 IEEE 6th International Conference on Advanced Computing (IACC).

[4]  Nasser Kehtarnavaz,et al.  Real-time robust vision-based hand gesture recognition using stereo images , 2013, Journal of Real-Time Image Processing.

[5]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Shuichi Kurabayashi,et al.  Monochrome glove: A robust real-time hand gesture recognition method by using a fabric glove with design of structured markers , 2016, 2016 IEEE Virtual Reality (VR).

[7]  Dan Xu,et al.  Online Dynamic Gesture Recognition for Human Robot Interaction , 2015, J. Intell. Robotic Syst..

[8]  Panos Nasiopoulos,et al.  A Random Forests-based approach for estimating depth of human body gestures using a single video camera , 2011, 2011 IEEE International Conference on Consumer Electronics (ICCE).

[9]  Mathias Kölsch,et al.  Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[10]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[11]  Ana-Maria Cretu,et al.  Static and Dynamic Hand Gesture Recognition in Depth Data Using Dynamic Time Warping , 2016, IEEE Transactions on Instrumentation and Measurement.

[12]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Mario Ciampi,et al.  Controller-free exploration of medical image data: Experiencing the Kinect , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[14]  Takayoshi Yamashita,et al.  Hand gesture based TV control system — Towards both user- & machine-friendly gesture applications , 2013, The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision.