Self-organizing maps for hand and full body tracking

Abstract Touch-free gesture technology opens new avenues for human–machine interaction. We show how self-organizing maps (SOM) can be used for hand and full body tracking. We use a range camera for data acquisition and apply a SOM-learning process for each frame in order to capture the pose. In a next step we introduce an extension of the SOM to 1D and 2D segments for an improved representation and skeleton tracking of body and hand. The proposed SOM based algorithms are very efficient and robust, and produce good tracking results. Their efficiency allows to implement these algorithms on embedded systems, which we demonstrate on an ARM-based embedded platform.

[1]  Thomas Martinetz,et al.  Self-Organizing Maps for Pose Estimation with a Time-of-Flight Camera , 2009, Dyn3D.

[2]  Eric Foxlin,et al.  Motion Tracking Requirements and Technologies , 2002 .

[3]  Luc Van Gool,et al.  Smart particle filtering for 3D hand tracking , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[4]  Chin-Chen Chang,et al.  Adaptive multiple sets of CSS features for hand posture recognition , 2006, Neurocomputing.

[5]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[6]  Kiyoshi Hoshino,et al.  3D Hand Pose Estimation Using a Single Camera for Unspecified Users , 2009, J. Robotics Mechatronics.

[7]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[8]  Stan Sclaroff,et al.  3D hand pose reconstruction using specialized mappings , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[9]  Björn Stenger,et al.  Hand Pose Estimation Using Hierarchical Detection , 2004, ECCV Workshop on HCI.

[10]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[11]  Thomas Martinetz,et al.  Deictic Gestures with a Time-of-Flight Camera , 2009, Gesture Workshop.

[12]  Stan Sclaroff,et al.  Estimating 3D hand pose from a cluttered image , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[13]  Achim Ebert,et al.  GPU Accelerated Gesture Detection for Real Time Interaction , 2007, Visualization of Large and Unstructured Data Sets.

[14]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[15]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[16]  Jakub Segen,et al.  Shadow gestures: 3D hand pose estimation using a single camera , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[17]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[18]  Michael Riis Andersen,et al.  Kinect Depth Sensor Evaluation for Computer Vision Applications , 2012 .

[19]  Marc M. Sebrechts,et al.  HANDBOOK OF VIRTUAL ENVIRONMENTS , 2014 .

[20]  HiltonAdrian,et al.  A survey of advances in vision-based human motion capture and analysis , 2006 .

[21]  Thomas Martinetz,et al.  Hand Tracking with an Extended Self-Organizing Map , 2012, WSOM.

[22]  Karan Singh,et al.  Eurographics/siggraph Symposium on Computer Animation (2003) Handrix: Animating the Human Hand , 2003 .

[23]  Yoshiaki Shirai,et al.  Real-time 3D hand posture estimation based on 2D appearance retrieval using monocular camera , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[24]  Wen-Kai Tai,et al.  Feature alignment approach for hand posture recognition based on curvature scale space , 2008, Neurocomputing.

[25]  Greg Mori,et al.  Real-time Motion-based Gesture Recognition Using the GPU , 2009, MVA.