Spectral Graph Skeletons for 3D Action Recognition

We present spectral graph skeletons (SGS), a novel graph-based method for action recognition from depth cameras. The contribution of this paper is to leverage a spectral graph wavelet transform (SGWT) for creating an overcomplete representation of an action signal lying on a 3D skeleton graph. The resulting SGS descriptor is efficiently computable in time linear in the action sequence length. We investigate the suitability of our method by experiments on three publicly available datasets, resulting in performance comparable to state-of-the-art action recognition approaches. Namely, our method achieves \(91.4\)% accuracy on the challenging MSRAction3D dataset in the cross-subject setting. SGS also achieves \(96.0\,\%\) and \(98.8\,\%\) accuracy on the MSRActionPairs3D and UCF-Kinect datasets, respectively. While this study focuses on action recognition, the proposed framework can in general be applied to any time series of graphs.

[1]  Pascal Frossard,et al.  Parametric dictionary learning for graph signals , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[2]  Hairong Qi,et al.  Group Sparsity and Geometry Constrained Dictionary Learning for Action Recognition from Depth Maps , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[4]  Kaspar Riesen,et al.  Recent advances in graph-based pattern recognition with applications in document analysis , 2011, Pattern Recognit..

[5]  Wanqing Li,et al.  Action recognition based on a bag of 3D points , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[6]  Michael Elad,et al.  Generalized Tree-Based Wavelet Transform , 2010, IEEE Transactions on Signal Processing.

[7]  Dimitri Van De Ville,et al.  Wavelet frames on graphs defined by fMRI functional connectivity , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[8]  Xiaodong Yang,et al.  EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[9]  Zicheng Liu,et al.  HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Cristian Sminchisescu,et al.  The Moving Pose: An Efficient 3D Kinematics Descriptor for Low-Latency Action Recognition and Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[12]  Mark Crovella,et al.  Graph wavelets for spatial traffic analysis , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[13]  Arthur D. Szlam,et al.  Diffusion wavelet packets , 2006 .

[14]  Quan Z. Sheng,et al.  Online human gesture recognition from motion data streams , 2013, ACM Multimedia.

[15]  Antonio Ortega,et al.  Spectral anomaly detection using graph-based filtering for wireless sensor networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Mario Fernando Montenegro Campos,et al.  STOP: Space-Time Occupancy Patterns for 3D Action Recognition from Depth Map Sequences , 2012, CIARP.

[17]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[18]  A. Sandryhaila,et al.  Nearest-neighbor image model , 2012, 2012 19th IEEE International Conference on Image Processing.

[19]  Ying Wu,et al.  Mining actionlet ensemble for action recognition with depth cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Zoubin Ghahramani,et al.  Graph Kernels by Spectral Transforms , 2006, Semi-Supervised Learning.

[21]  James H. Garrett,et al.  Semi-Supervised Multiresolution Classification Using Adaptive Graph Filtering With Application to Indirect Bridge Structural Health Monitoring , 2014, IEEE Transactions on Signal Processing.

[22]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[23]  Sunil K. Narang,et al.  Graph-wavelet filterbanks for edge-aware image processing , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[24]  Xiaodong Yang,et al.  Recognizing actions using depth motion maps-based histograms of oriented gradients , 2012, ACM Multimedia.

[25]  Sunil K. Narang,et al.  Graph based transforms for depth video coding , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[27]  Marwan Torki,et al.  Histogram of Oriented Displacements (HOD): Describing Trajectories of Human Joints for Action Recognition , 2013, IJCAI.

[28]  Joseph J. LaViola,et al.  Exploring the Trade-off Between Accuracy and Observational Latency in Action Recognition , 2013, International Journal of Computer Vision.

[29]  Ying Wu,et al.  Learning Maximum Margin Temporal Warping for Action Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[30]  Sunil K. Narang,et al.  Perfect Reconstruction Two-Channel Wavelet Filter Banks for Graph Structured Data , 2011, IEEE Transactions on Signal Processing.

[31]  Kaspar Riesen,et al.  Towards the unification of structural and statistical pattern recognition , 2012, Pattern Recognit. Lett..

[32]  Devdatt P. Dubhashi,et al.  Entity disambiguation in anonymized graphs using graph kernels , 2013, CIKM.

[33]  Pascal Frossard,et al.  Chebyshev polynomial approximation for distributed signal processing , 2011, 2011 International Conference on Distributed Computing in Sensor Systems and Workshops (DCOSS).

[34]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[35]  R. Coifman,et al.  Diffusion Wavelets , 2004 .

[36]  Pascal Frossard,et al.  Inference of mobility patterns via Spectral Graph Wavelets , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Jake K. Aggarwal,et al.  View invariant human action recognition using histograms of 3D joints , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[38]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[39]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Michael G. Rabbat,et al.  Approximating signals supported on graphs , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).