Recognition of signed dynamic expressions observed by ToF camera

Time-of-flight (ToF) cameras acquire 3D information about observed scenes. They are increasingly used for hand gesture recognition. This paper is also related to this problem. In contrast to other, hand segmentation based, works we propose using point cloud processing and the Viewpoint Feature Histogram (VFH) as the global descriptor of the scene. To empower the distinctiveness of the descriptor a modification is proposed which consists in dividing the work space into smaller cells and calculating the VFH for each of them. The method is applied to chosen dynamic gestures of the Polish sign language. The gestures are relatively difficult to recognise because the hands often are not the objects nearest the camera and/or touch each other, touch the head or appear in the background of the face. Results of ten-fold cross-validation for the nearest neighbour classifier based on dynamic time warping for sequence comparison are given to justify the proposed approach.

[1]  Filiberto Chiabrando,et al.  SR-4000 TOF camera: further experimental tests and first applications to metric survey , 2010 .

[2]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Luc Van Gool,et al.  Real-time sign language letter and word recognition from depth data , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[4]  Mariusz Oszust,et al.  Recognition of signed expressions observed by Kinect Sensor , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[5]  Rui Min Hu,et al.  Real-Time 3D Hand Tracking from Depth Images , 2013 .

[6]  Morteza Zahedi,et al.  Robust Sign Language Recognition System Using ToF Depth Cameras , 2011, ArXiv.

[7]  Ralf Dörner,et al.  Recognition of Dynamic Hand Gestures with Time-of-Flight Cameras , 2010 .

[8]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[9]  Nico Blodow,et al.  Learning informative point classes for the acquisition of object model maps , 2008, 2008 10th International Conference on Control, Automation, Robotics and Vision.

[10]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[11]  Jesús Bescós,et al.  Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models , 2011, Machine Vision and Applications.

[12]  Eamonn J. Keogh,et al.  Three Myths about Dynamic Time Warping Data Mining , 2005, SDM.

[13]  L. Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[14]  Luc Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, WACV.

[15]  Serban Oprisescu,et al.  Automatic static hand gesture recognition using ToF cameras , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).