Hand pose recognition in First Person Vision through graph spectral analysis

With the growing availability of wearable technology, video recording devices have become so intimately tied to individuals, that they are able to record the movements of users' hands, making hand-based applications one the most explored area in First Person Vision (FPV). In particular, hand pose recognition plays a fundamental role in tasks such as gesture and activity recognition, which in turn represent the base for developing human-machine interfaces or augmented reality applications. In this work we propose a graph-based representation of hands seen from the point of view of the user, obtained through the shape-fitting capability of a modified Instantaneous Topological Map. Spectral analysis of the graph Laplacian allows to arrange eigenvalues in vectors of features, which prove to be discriminative in classifying the considered hand poses.

[1]  Walterio W. Mayol-Cuevas,et al.  High level activity recognition using low resolution wearable vision , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[2]  Aaron M. Dollar,et al.  Finding small, versatile sets of human grasps to span common objects , 2013, 2013 IEEE International Conference on Robotics and Automation.

[3]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[4]  Hironobu Takagi,et al.  Recognizing hand-object interactions in wearable camera videos , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[5]  Cheng Li,et al.  Pixel-Level Hand Detection in Ego-centric Videos , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Cheng Li,et al.  Model Recommendation with Virtual Probes for Egocentric Hand Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Nuno Vasconcelos,et al.  Robust Deformable and Occluded Object Tracking With Dynamic Graph , 2014, IEEE Transactions on Image Processing.

[8]  Ali Farhadi,et al.  Understanding egocentric activities , 2011, 2011 International Conference on Computer Vision.

[9]  Matthias Rauterberg,et al.  Left/Right Hand Segmentation in Egocentric Videos , 2016, 1607.06264.

[10]  Antonio Ortega,et al.  Towards a sampling theorem for signals on arbitrary graphs , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Yoichi Sato,et al.  A scalable approach for understanding the visual structures of hand grasps , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Sarfraz Khurshid,et al.  Dynamic Shape Analysis Using Spectral Graph Properties , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[13]  Helge J. Ritter,et al.  An instantaneous topological mapping model for correlated stimuli , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[14]  Matthias Rauterberg,et al.  Towards a unified framework for hand-based methods in First Person Vision , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[15]  Rita Cucchiara,et al.  Hand segmentation for gesture recognition in EGO-vision , 2013, IMMPD '13.

[16]  Carlo S. Regazzoni,et al.  A generative superpixel method , 2014, 17th International Conference on Information Fusion (FUSION).

[17]  Kris M. Kitani,et al.  How do we use our hands? Discovering a diverse set of common grasps , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Matthias Rauterberg,et al.  The Evolution of First Person Vision Methods: A Survey , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Matthias Rauterberg,et al.  A Dynamic Approach and a New Dataset for Hand-detection in First Person Vision , 2015, CAIP.

[20]  Nikos Papamarkos,et al.  Hand gesture recognition using a neural network shape fitting technique , 2009, Eng. Appl. Artif. Intell..

[21]  Carlo S. Regazzoni,et al.  Optimizing Superpixel Clustering for Real-Time Egocentric-Vision Applications , 2015, IEEE Signal Processing Letters.

[22]  Carlo S. Regazzoni,et al.  Hand detection in First Person Vision , 2013, Proceedings of the 16th International Conference on Information Fusion.