A Unified Fuzzy Framework for Human-Hand Motion Recognition

Unconstrained human-hand motions that consist grasp motions and in-hand manipulations lead to a fundamental challenge that many algorithms have to face in both theoretical and practical development, mainly due to the complexity and dexterity of the human hand. There is no effective solution reported to recognize in-hand manipulations, although recognition algorithms have been proposed to recognize grasp motions in constrained scenarios. This paper proposes a novel unified fuzzy framework of a set of recognition algorithms: time clustering, fuzzy active axis Gaussian mixture mode, and fuzzy empirical copula, from numerical clustering to data dependence structure in the context of optimally real-time human-hand motion recognition. Time clustering is a fuzzy time-modeling approach that is based on fuzzy clustering and Takagi--Sugeno modeling with a numerical value as output. The fuzzy active axis Gaussian mixture model effectively extract abstract Gaussian pattern to represent components of hand gestures with a fast convergence. A fuzzy empirical copula utilizes the dependence structure among the finger joint angles to recognize the motion type. The proposed algorithms have been evaluated on a wide range of scenarios of human-hand recognition: 1) datasets that include 13 grasps and ten in-hand manipulations; 2) single subject and multiple subjects; and 3) varying training samples. The experimental results have demonstrated that the proposed framework outperforms the hidden Markov model (HMM) and Gaussian mixture model in terms of both effectiveness and efficiency criteria.

[1]  Benjamín R. C. Bedregal,et al.  Fuzzy Rule-Based Hand Gesture Recognition , 2006, IFIP AI.

[2]  Toshiaki Ejima,et al.  Real-Time hand Gesture Recognition Using Pseudo 3-D Hidden Markov Model , 2006, 2006 5th IEEE International Conference on Cognitive Informatics.

[3]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Tim J. Ellis,et al.  Recognizing hand gesture using Fourier descriptors , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[5]  W. Kruskal.  Ordinal Measures of Association , 1958 .

[6]  Bhiksha Raj,et al.  One-handed gesture recognition using ultrasonic Doppler sonar , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Honghai Liu,et al.  Fast estimating data dependence structure via fuzzy empirical copula , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[8]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[9]  Honghai Liu,et al.  A Fuzzy Qualitative Framework for Connecting Robot Qualitative and Quantitative Representations , 2008, IEEE Transactions on Fuzzy Systems.

[10]  R. Zollner,et al.  Dynamic grasp recognition within the framework of programming by demonstration , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[11]  Rainer Palm,et al.  Recognition of human grasps by time-clustering and fuzzy modeling , 2009, Robotics Auton. Syst..

[12]  Bill Ravens,et al.  An Introduction to Copulas , 2000, Technometrics.

[13]  Antonio Bicchi,et al.  Hands for dexterous manipulation and robust grasping: a difficult road toward simplicity , 2000, IEEE Trans. Robotics Autom..

[14]  Hong Liu,et al.  FPGA based hardware architecture for HIT/DLR hand , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Toshiaki Ejima,et al.  Real-Time Hand Tracking and Gesture Recognition System , 2005 .

[16]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[17]  Yoichi Sato,et al.  Real-time input of 3D pose and gestures of a user's hand and its applications for HCI , 2001, Proceedings IEEE Virtual Reality 2001.

[18]  Adrian E. Raftery,et al.  How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis , 1998, Comput. J..

[19]  Aditya Ramamoorthy,et al.  Recognition of dynamic hand gestures , 2003, Pattern Recognit..

[20]  E. Lehmann.  Some Concepts of Dependence , 1966 .

[21]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[22]  Agnès Just,et al.  A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition , 2009, Comput. Vis. Image Underst..

[23]  Rainer Palm,et al.  Programming-by-Demonstration of reaching motions - A next-state-planner approach , 2010, Robotics Auton. Syst..

[24]  C M Light,et al.  Development of a lightweight and adaptable multiple-axis hand prosthesis. , 2000, Medical engineering & physics.

[25]  A. Raftery,et al.  Detecting features in spatial point processes with clutter via model-based clustering , 1998 .

[26]  Danica Kragic,et al.  Modeling and recognition of actions through motor primitives , 2008, 2008 IEEE International Conference on Robotics and Automation.

[27]  Thea Iberall,et al.  Human Prehension and Dexterous Robot Hands , 1997, Int. J. Robotics Res..

[28]  Silvestro Micera,et al.  Design of a cybernetic hand for perception and action , 2006, Biological Cybernetics.

[29]  Ignazio Infantino,et al.  A posture sequence learning system for an anthropomorphic robotic hand , 2004, Robotics Auton. Syst..

[30]  Honghai Liu,et al.  Applying fuzzy EM algorithm with a fast convergence to GMMs , 2010, International Conference on Fuzzy Systems.

[31]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[32]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[33]  Pedram Afshar,et al.  On the design of robotic hands for brain-machine interface. , 2006, Neurosurgical focus.

[34]  Rüdiger Dillmann,et al.  Towards Cognitive Robots: Building Hierarchical Task Representations of Manipulations from Human Demonstration , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[35]  Gianluca Palli,et al.  Development of UB Hand 3: Early Results , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[36]  Katsushi Ikeuchi,et al.  A sensor fusion approach for recognizing continuous human grasping sequences using hidden Markov models , 2005, IEEE Transactions on Robotics.

[37]  Heni Ben Amor,et al.  Grasp Recognition with Uncalibrated Data Gloves - A Comparison of Classification Methods , 2007, 2007 IEEE Virtual Reality Conference.

[38]  Xiangyang Zhu,et al.  Dynamic Grasp Recognition Using Time Clustering, Gaussian Mixture Models and Hidden Markov Models , 2009, Adv. Robotics.

[39]  Latifur Khan,et al.  Real-time classification of variable length multi-attribute motions , 2006, Knowledge and Information Systems.

[40]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[41]  Paul F. M. J. Verschure,et al.  Environmentally mediated synergy between perception and behaviour in mobile robots , 2003, Nature.

[42]  Honghai Liu,et al.  Fuzzy Qualitative Robot Kinematics , 2008, IEEE Transactions on Fuzzy Systems.

[43]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[44]  Nick Hawes,et al.  Crossmodal content binding in information-processing architectures , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[45]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[46]  Honghai Liu,et al.  Exploring Human Hand Capabilities Into Embedded Multifingered Object Manipulation , 2011, IEEE Transactions on Industrial Informatics.

[47]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[48]  Honghai Liu,et al.  Recognizing Hand Grasp and Manipulation Through Empirical Copula , 2010, Int. J. Soc. Robotics.

[49]  Danica Kragic,et al.  Action recognition and understanding through motor primitives , 2007, Adv. Robotics.

[50]  Changshui Zhang,et al.  Active curve axis Gaussian mixture models , 2005, Pattern Recognit..

[51]  Rainer Palm,et al.  Recognition and Teaching of Robot Skills by Fuzzy Time-Modeling , 2009, IFSA/EUSFLAT Conf..

[52]  Geoffrey E. Hinton,et al.  Glove-TalkII-a neural-network interface which maps gestures to parallel formant speech synthesizer controls , 1997, IEEE Trans. Neural Networks.

[53]  Rainer Palm,et al.  Perception modeling for human-like artificial sensor systems , 2007, Int. J. Hum. Comput. Stud..