Real-time hand posture recognition using range data

A hand posture recognition system using 3D data is described. The system relies on a novel 3D sensor that generates a dense range image of the scene. The main advantage of the proposed system, compared to other gesture recognition techniques, is the capability for robust unconstrained recognition of complex hand postures such as those encountered in sign language alphabets. This is achieved by explicitly utilizing 3D hand geometry. Moreover, the proposed approach does not rely on color information, and guarantees robust segmentation of the hand under varying illumination conditions, and scene content. Several novel 3D image analysis algorithms are presented, covering the complete processing chain: 3D image acquisition, arm segmentation, hand-forearm segmentation, hand pose estimation, 3D feature extraction, and gesture classification. The proposed system is extensively evaluated.

[1]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[2]  Nebojsa Jojic,et al.  Detection and estimation of pointing gestures in dense disparity maps , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[3]  Björn Stenger,et al.  Estimating 3D hand pose using hierarchical multi-label classification , 2007, Image Vis. Comput..

[4]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[5]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Anil K. Jain,et al.  Feature extraction methods for character recognition-A survey , 1996, Pattern Recognit..

[7]  Nicol N. Schraudolph,et al.  3D hand tracking by rapid stochastic gradient descent using a skinning model , 2004 .

[8]  Edoardo Charbon,et al.  3D Hand Model Fitting for Virtual Keyboard System , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[9]  Ming Xie,et al.  Finger identification and hand posture recognition for human-robot interaction , 2007, Image Vis. Comput..

[10]  Danijel Skocaj,et al.  Robust recognition and pose determination of 3-D objects using range images in eigenspace approach , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[11]  David G. Stork,et al.  Pattern Classification , 1973 .

[12]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[13]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[14]  Olivier D. Faugeras,et al.  Finding pose of hand in video images: a stereo-based approach , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[15]  Mircea Nicolescu,et al.  A Review on Vision-Based Full DOF Hand Motion Estimation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[16]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[17]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[18]  Bernd Deimel,et al.  Improving Hand-Gesture Recognition via Video Based Methods for the Separation of the Forearm from th , 1999 .

[19]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[20]  Stan Sclaroff,et al.  Estimating 3D hand pose from a cluttered image , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  Michael G. Strintzis,et al.  Tracking the left ventricle in echocardiographic images by learning heart dynamics , 1999, IEEE Transactions on Medical Imaging.

[22]  Patrick J. Flynn,et al.  A Survey Of Free-Form Object Representation and Recognition Techniques , 2001, Comput. Vis. Image Underst..

[23]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[24]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[25]  Haiying Guan,et al.  Model-based 3D hand posture estimation from a single 2D image , 2002, Image Vis. Comput..

[26]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[27]  Gary R. Bradski,et al.  Stereo based gesture recognition invariant to 3D pose and lighting , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Michael G. Strintzis,et al.  Real-time acquisition of depth and color images using structured light and its application to 3D face recognition , 2005, Real Time Imaging.

[29]  Rogério Schmidt Feris,et al.  The isometric self-organizing map for 3D hand pose estimation , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[30]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Akira Utsumi,et al.  Multiple-hand-gesture tracking using multiple cameras , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[32]  Andrew E. Johnson,et al.  Surface matching for object recognition in complex three-dimensional scenes , 1998, Image Vis. Comput..

[33]  C. Jennings,et al.  Robust finger tracking with multiple cameras , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[34]  Björn Stenger,et al.  Hand Pose Estimation Using Hierarchical Detection , 2004, ECCV Workshop on HCI.

[35]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[36]  Thomas S. Huang,et al.  Static Hand Gesture Recognition based on Local Orientation Histogram Feature Distribution Model , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[37]  Sam T. Roweis,et al.  EM Algorithms for PCA and SPCA , 1997, NIPS.

[38]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[39]  Kazunori Umeda,et al.  Recognition of hand gestures using range images , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[40]  Chin Seng Chua,et al.  Point Signatures: A New Representation for 3D Object Recognition , 1997, International Journal of Computer Vision.