Action Recognition Online with Hierarchical Self-Organizing Maps

We present a hierarchical self-organizing map based system for online recognition of human actions. We have made a first evaluation of our system by training it on two different sets of recorded human actions, one set containing manner actions and one set containing result actions, and then tested it by letting a human performer carry out the actions online in real time in front of the system's 3D-camera. The system successfully recognized more than 94% of the manner actions and most of the result actions carried out by the human performer.

[1]  Wanqing Li,et al.  Action recognition based on a bag of 3D points , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[2]  Marco La Cascia,et al.  Gesture Modeling by Hanklet-Based Hidden Markov Model , 2014, ACCV.

[3]  Peter Gärdenfors,et al.  Event structure, conceptual spaces and the semantics of verbs , 2012 .

[4]  Meinard Müller,et al.  Motion templates for automatic classification and retrieval of motion capture data , 2006, SCA '06.

[5]  Peter Gärdenfors,et al.  Using Conceptual Spaces to Model Actions and Events , 2012, J. Semant..

[6]  Joseph J. LaViola,et al.  Exploring the Trade-off Between Accuracy and Observational Latency in Action Recognition , 2013, International Journal of Computer Vision.

[7]  Sinan Kalkan,et al.  Verb concepts from affordances , 2014 .

[8]  Christian Balkenius,et al.  Associative Self-organizing Map , 2009, IJCCI.

[9]  Magnus Johnsson,et al.  Recognizing actions with the associative self-organizing map , 2013, 2013 XXIV International Conference on Information, Communication and Automation Technologies (ICAT).

[10]  Magnus Johnsson,et al.  Internal Simulation of an Agent's Intentions , 2012, BICA.

[11]  Beth Levin,et al.  Argument Realization: Acknowledgments , 2005 .

[12]  S. Runeson,et al.  Kinematic specification of dynamics as an informational basis for person and action perception: Expe , 1983 .

[13]  L. Vaina.  From shapes and movements to objects and actions , 2004, Synthese.

[14]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[15]  Magnus Johnsson,et al.  Discriminating and simulating actions with the associative self-organising map , 2015, Connect. Sci..

[16]  Sergio Escalera,et al.  Gesture and Action Recognition by Evolved Dynamic Subgestures , 2015, BMVC.

[17]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[18]  Ramakant Nevatia,et al.  Recognition and Segmentation of 3-D Human Action Using HMM and Multi-class AdaBoost , 2006, ECCV.

[19]  Tinne Tuytelaars,et al.  Towards sign language recognition based on body parts relations , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[20]  Peter Ford Dominey,et al.  Linking Language with Embodied and Teleological Representations of Action for Humanoid Cognition , 2010, Front. Neurorobot..

[21]  Magnus Johnsson,et al.  Hierarchies of Self-Organizing Maps for action recognition , 2016, Cognitive Systems Research.

[22]  Magnus Johnsson,et al.  Simulating Actions with the Associative Self-Organizing Map , 2013, AIC@AI*IA.

[23]  W. Frawley.  Mind as Action , 1998, Trends in Cognitive Sciences.

[24]  D. Marr,et al.  Representation and recognition of the movements of shapes , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[25]  Christian Balkenius,et al.  Ikaros: Building cognitive models for robots , 2010, Adv. Eng. Informatics.

[26]  Giulio Sandini,et al.  The ITALK project: Integration and Transfer of Action and Language Knowledge in Robots , 2008 .

[27]  E. Rosch.  Cognitive Representations of Semantic Categories. , 1975 .

[28]  Ying Wu,et al.  Mining actionlet ensemble for action recognition with depth cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Ilya Sutskever,et al.  Learning Recurrent Neural Networks with Hessian-Free Optimization , 2011, ICML.

[30]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[31]  G. Hesslow.  Conscious thought as simulation of behaviour and perception , 2002, Trends in Cognitive Sciences.

[32]  Ying Wu,et al.  Robust 3D Action Recognition with Random Occupancy Patterns , 2012, ECCV.

[33]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[34]  Peter Gärdenfors,et al.  Conceptual spaces - the geometry of thought , 2000 .

[35]  Zicheng Liu,et al.  HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Peter Gärdenfors,et al.  Representing actions and functional properties in conceptual spaces , 2007 .

[37]  Xiaodong Yang,et al.  Recognizing actions using depth motion maps-based histograms of oriented gradients , 2012, ACM Multimedia.

[38]  Magnus Johnsson,et al.  Action Recognition based on Hierarchical Self-Organizing Maps , 2014, AIC.