Trajectory Analysis on Spherical Self-Organizing Maps with Application to Gesture Recognition

We propose a new approach to gesture recognition using the properties of Spherical Self-Organizing Map (SSOM). Unbounded mapping of data onto a SSOM creates not only a powerful tool for visualization but also for modeling spatiotemporal information of gesture data. Once mapped onto a SSOM the gesture data is treated as a series of postures. A set of postures describing a specific path on the SSOM for a gesture is used as a trajectory. Although some trajectories may share the same postures, the path consisting of posture transitions will always be unique. Different variations of posture transitions occurring within a gesture trajectory are used to classify new unknown gestures. Experimental results on datasets involving full body and hand gestures show the effectiveness of our proposed method.

[1]  Wing W. Y. Ng,et al.  Content-based image retrieval using color moment and Gabor texture feature , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[2]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[3]  Yoshua Bengio,et al.  Neural networks for speech and sequence recognition , 1996 .

[4]  Masaki Oshita,et al.  Automatic Learning of Gesture Recognition Model Using SOM and SVM , 2010, ISVC.

[5]  C. Won,et al.  Efficient Use of MPEG‐7 Edge Histogram Descriptor , 2002 .

[6]  Marc M. Van Hulle,et al.  Comparison of Flat SOM with Spherical SOM. A Case Study , 2007 .

[7]  Kyle Johnsen,et al.  Immersive virtual reality on-the-go , 2013, 2013 IEEE Virtual Reality (VR).

[8]  Andreas Nürnberger,et al.  Summarizing Video Information Using Self-Organizing Maps , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[9]  Mingjing Li,et al.  Color texture moments for content-based image retrieval , 2002, Proceedings. International Conference on Image Processing.

[10]  D. Kammer,et al.  Taxonomy and Overview of Multi-touch Frameworks: Architecture, Scope and Features , 2010 .

[11]  Jungpil Shin,et al.  Dynamic Positional Warping: Dynamic Time Warping for Online Handwriting , 2009, Int. J. Pattern Recognit. Artif. Intell..

[12]  Joseph J. LaViola,et al.  Immersive VR for Scientific Visualization: A Progress Report , 2000, IEEE Computer Graphics and Applications.

[13]  Yong Man Ro,et al.  MPEG-7 Texture Descriptors , 2001, Int. J. Image Graph..

[14]  Markus H. Gross,et al.  Visualization of multidimensional image data sets using a neural network , 1993, The Visual Computer.

[15]  A. Kendon.  Some Relationships Between Body Motion and Speech , 1972 .

[16]  Mathias Lux,et al.  SpCD - Spatial Color Distribution Descriptor - A Fuzzy Rule based Compact Composite Descriptor Appropriate for Hand Drawn Color Sketches Retrieval , 2016, ICAART.

[17]  Cordelia Schmid,et al.  Actions in context , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Atsushi Shimada,et al.  Gesture recognition using sparse code of Hierarchical SOM , 2008, 2008 19th International Conference on Pattern Recognition.

[19]  Jorma Laaksonen,et al.  PicSOM Experiments in TRECVID 2018 , 2015, TRECVID.

[20]  Christos Pateritsas,et al.  Hand trajectory based gesture recognition using self-organizing feature maps and markov models , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[21]  Andreas Nürnberger,et al.  Weighted Self-Organizing Maps: Incorporating User Feedback , 2003, ICANN.

[22]  Christos Pateritsas,et al.  Probabilistic Video-Based Gesture Recognition Using Self-organizing Feature Maps , 2007, ICANN.

[23]  Paul Over,et al.  The trecvid 2007 BBC rushes summarization evaluation pilot , 2007, TVS '07.

[24]  Jorma Laaksonen,et al.  Video Segmentation and Shot Boundary Detection Using Self-Organizing Maps , 2007, SCIA.

[25]  Jouko Lampinen,et al.  Clustering properties of hierarchical self-organizing maps , 1992, Journal of Mathematical Imaging and Vision.

[26]  Claude Sammut,et al.  Classification of Multivariate Time Series and Structured Data Using Constructive Induction , 2005, Machine Learning.

[27]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[28]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[29]  Dim P. Papadopoulos,et al.  Video Summarization Using a Self-Growing and Self-Organized Neural Gas Network , 2011, MIRAGE.

[30]  Robert B. Fisher,et al.  Incremental One-Class Learning with Bounded Computational Complexity , 2007, ICANN.

[31]  Stanley C. Ahalt,et al.  Neural Networks for Vector Quantization of Speech and Images , 1990, IEEE J. Sel. Areas Commun..

[32]  Kostas Karpouzis,et al.  Manual annotation and automatic image processing of multimodal emotional behaviors: validating the annotation of TV interviews , 2007, Personal and Ubiquitous Computing.

[33]  Mohammed Waleed Kadous,et al.  Temporal classification: extending the classification paradigm to multivariate time series , 2002 .

[34]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[35]  Borko Furht,et al.  Encyclopedia of Multimedia , 2006 .

[36]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[37]  Yiannis S. Boutalis,et al.  Content based radiology image retrieval using a fuzzy rule based scalable composite descriptor , 2009, Multimedia Tools and Applications.

[38]  H. Ritter.  Self-Organizing Maps on non-euclidean Spaces , 1999 .

[39]  Leszek Cieplinski.  MPEG-7 Color Descriptors and Their Applications , 2001, CAIP.

[40]  Yiu-ming Cheung,et al.  k*-Means: A new generalized k-means clustering algorithm , 2003, Pattern Recognit. Lett..

[41]  Yiannis S. Boutalis,et al.  Accurate Image Retrieval Based on Compact Composite Descriptors and Relevance Feedback Information , 2010, Int. J. Pattern Recognit. Artif. Intell..

[42]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[43]  Jorma Laaksonen,et al.  Rushes summarization with self-organizing maps , 2007, TVS '07.

[44]  ARCHANA P. SANGOLE,et al.  Spherical Self-Organizing Feature Map: an introductory Review , 2006, Int. J. Bifurc. Chaos.