Hardware implementation for a hand recognition system on FPGA

This paper presents the implementation for hand tacking and gesture recognition on FPGA. For hand tracking, an improved hand segmentation algorithm combined YCbCr color space and region growing algorithm is used for valid region separation, and then morphological operations including erosion and dilation are applied to this system in order to remove extra noise. The three-frame-difference motion detection method is used to detect moving hand. Finally the centroid of hand can be located. For gesture recognition, we develop a hand model based on finger state, converting pose recognition to finger state detection. The hand recognition system has been entirely implemented on Cyclone II FPGA.

[1]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[2]  Akira Utsumi,et al.  Multiple-hand-gesture tracking using multiple cameras , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Pham The Bao,et al.  A New Approach to Hand Tracking and Gesture Recognition by a New Feature Type and HMM , 2009, 2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery.

[4]  Yea-shuan Huang,et al.  An Occlusion-Resolving Hand Tracking Method , 2014, 2014 7th International Conference on Ubi-Media Computing and Workshops.

[5]  Shahrel Azmin Suandi,et al.  Hand gesture tracking system using Adaptive Kalman Filter , 2010, 2010 10th International Conference on Intelligent Systems Design and Applications.

[6]  Junaidi Abdullah,et al.  A new method for hand segmentation using free-form skin color model , 2010, 2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE).

[7]  Shawmin Lei,et al.  Real-time hand tracking on depth images , 2011, 2011 Visual Communications and Image Processing (VCIP).

[8]  Junwei Gao,et al.  Robot fish detection based on a combination method of three-frame-difference and background subtraction , 2014, CCDC 2014.

[9]  Tieniu Tan,et al.  Real time hand tracking by combining particle filtering and mean shift , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[10]  Roger J. Hubbold,et al.  Real-time Hand Tracking With Variable-Length Markov Models of Behaviour , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[11]  L. Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[12]  Supavadee Aramvith,et al.  Improved face and hand tracking for sign language recognition , 2005, International Conference on Information Technology: Coding and Computing (ITCC'05) - Volume II.

[13]  Helge J. Ritter,et al.  Real-time hand tracking using synergistic inverse kinematics , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Che-Hao Chang,et al.  Improved Hand Tracking System , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Gabriel Pérez,et al.  Explicit image detection using YCbCr space color model as skin detection , 2011 .

[16]  Ming Che,et al.  A Hardware/Software Co-design of a Face Detection Algorithm Based on FPGA , 2010, 2010 International Conference on Measuring Technology and Mechatronics Automation.

[17]  Yue Cao,et al.  Research on a Skin Color Detection Algorithm Based on Self-adaptive Skin Color Model , 2010, 2010 International Conference on Communications and Intelligence Information Security.

[18]  Yoichi Sato,et al.  Real-time input of 3D pose and gestures of a user's hand and its applications for HCI , 2001, Proceedings IEEE Virtual Reality 2001.

[19]  Xiaofen Xing,et al.  Hand gesture segmentation based on improved kalman filter and TSL skin color model , 2011, 2011 International Conference on Multimedia Technology.

[20]  Miroslaw Bober,et al.  A low cost FPGA system for high speed face detection and tracking , 2003, Proceedings. 2003 IEEE International Conference on Field-Programmable Technology (FPT) (IEEE Cat. No.03EX798).

[21]  Ulrich Neumann,et al.  Real-time Hand Pose Recognition Using Low-Resolution Depth Images , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Wen-Hui Chen,et al.  Implementation of Low-Cost Vision-Based Gesture Recognition Systems Based on FPGA Approach , 2012, 2012 International Symposium on Computer, Consumer and Control.

[23]  Roger J. Hubbold,et al.  A real-time hand tracker using variable-length Markov models of behaviour , 2007, Comput. Vis. Image Underst..

[24]  Wang Yaowei,et al.  Robot fish detection based on a combination method of three-frame-difference and background subtraction , 2014, The 26th Chinese Control and Decision Conference (2014 CCDC).

[25]  Santolo Meo,et al.  World Scientific and Engineering Academy and Society (WSEA) , 2010 .

[26]  Yangsheng Xu,et al.  Hand tracking and pose recognition via depth and color information , 2012, 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[27]  Rini Akmeliawati,et al.  A hybrid method using haar-like and skin-color algorithm for hand posture detection, recognition and tracking , 2010, 2010 IEEE International Conference on Mechatronics and Automation.

[28]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Tsuhan Chen,et al.  Hand tracking using spatial gesture modeling and visual feedback for a virtual DJ system , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[30]  Yang Li,et al.  A real-time multi-cue hand tracking algorithm based on computer vision , 2010, 2010 IEEE Virtual Reality Conference (VR).

[31]  Melanie Po-Leen Ooi.  Hardware implementation for face detection on Xilinx Virtex-II FPGA using the reversible component transformation colour space , 2006, Third IEEE International Workshop on Electronic Design, Test and Applications (DELTA'06).

[32]  Xuan-Thuan Nguyen,et al.  A real-time DSP-based hand gesture recognition system , 2012, 2012 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[33]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Wei Wang,et al.  Hand segmentation using skin color and background information , 2012, 2012 International Conference on Machine Learning and Cybernetics.