View-Invariant Action Recognition Based on Artificial Neural Networks

In this paper, a novel view invariant action recognition method based on neural network representation and recognition is proposed. The novel representation of action videos is based on learning spatially related human body posture prototypes using self organizing maps. Fuzzy distances from human body posture prototypes are used to produce a time invariant action representation. Multilayer perceptrons are used for action classification. The algorithm is trained using data from a multi-camera setup. An arbitrary number of cameras can be used in order to recognize actions using a Bayesian framework. The proposed method can also be applied to videos depicting interactions between humans, without any modification. The use of information captured from different viewing angles leads to high classification performance. The proposed method is the first one that has been tested in challenging experimental setups, a fact that denotes its effectiveness to deal with most of the open issues in action recognition.

[1]  Alexandros Iosifidis,et al.  Multi-view human movement recognition based on fuzzy distances and linear discriminant analysis , 2012, Comput. Vis. Image Underst..

[2]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[3]  Amit K. Roy-Chowdhury,et al.  Towards A Multi-Terminal Video Compression Algorithm By Integrating Distributed Source Coding With Geometrical Constraints , 2007, J. Multim..

[4]  Steven K. Feiner,et al.  User interface management techniques for collaborative mobile augmented reality , 2001, Comput. Graph..

[5]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[6]  Alexandros Iosifidis,et al.  Activity-Based Person Identification Using Fuzzy Representation and Discriminant Learning , 2012, IEEE Transactions on Information Forensics and Security.

[7]  Osama Masoud,et al.  A method for human action recognition , 2003, Image Vis. Comput..

[8]  Mohiuddin Ahmad,et al.  HMM-based Human Action Recognition Using Multiview Image Sequences , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[9]  Yang Wang,et al.  Hidden Part Models for Human Action Recognition: Probabilistic versus Max Margin , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Michael Schmitt,et al.  Self-organization of spiking neurons using action potential timing , 1998, IEEE Trans. Neural Networks.

[11]  Rémi Ronfard,et al.  Action Recognition from Arbitrary Views using 3D Exemplars , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  Ioannis Pitas,et al.  The i3DPost Multi-View and 3D Human Action/Interaction Database , 2009, 2009 Conference for Visual Media Production.

[13]  Jesse Hoey,et al.  Representation and recognition of complex human motion , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Peter H. N. de With,et al.  Flexible Human Behavior Analysis Framework for Video Surveillance Applications , 2010, Int. J. Digit. Multim. Broadcast..

[15]  Mohiuddin Ahmad,et al.  Human action recognition using shape and CLG-motion flow from multi-view image sequences , 2008, Pattern Recognit..

[16]  James Noble,et al.  Video game values: Human-computer interaction and games , 2007, Interact. Comput..

[17]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Thomas Serre,et al.  A Biologically Inspired System for Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[20]  Peyman Milanfar,et al.  Action Recognition from One Example , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ronen Basri,et al.  Actions as Space-Time Shapes , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Ioannis Pitas,et al.  View indepedent human movement recognition from multi-view video exploiting a circular invariant posture representation , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[23]  Oswald Lanz,et al.  Approximate Bayesian multibody tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Aurélie Bugeau,et al.  Tracking with Occlusions via Graph Cuts , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Mauro Ursino,et al.  Recognition of Abstract Objects Via Neural Oscillators: Interaction Among Topological Organization, Associative Memory and Gamma Band Synchronization , 2009, IEEE Transactions on Neural Networks.

[26]  Rémi Ronfard,et al.  Free viewpoint action recognition using motion history volumes , 2006, Comput. Vis. Image Underst..

[27]  Michael Schmitt,et al.  On the sample complexity of learning for networks of spiking neurons with nonlinear synaptic interactions , 2001, IEEE Transactions on Neural Networks.

[28]  D. Terzopoulos,et al.  Surveillance camera scheduling: a virtual vision approach , 2005, VSSN@MM.

[29]  Ramakant Nevatia,et al.  Single View Human Action Recognition using Key Pose Matching and Viterbi Path Searching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  T. Poggio,et al.  Cognitive neuroscience: Neural mechanisms for the recognition of biological movements , 2003, Nature Reviews Neuroscience.

[31]  Du Tran,et al.  Human Activity Recognition with Metric Learning , 2008, ECCV.

[32]  Anastasios Tefas,et al.  Combining Fuzzy Vector Quantization With Linear Discriminant Analysis for Continuous Human Movement Recognition , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[34]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Hélène Laurent,et al.  Review and evaluation of commonly-implemented background subtraction algorithms , 2008, 2008 19th International Conference on Pattern Recognition.

[36]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[37]  Tieniu Tan,et al.  Modelling the Effect of View Angle Variation on Appearance-Based Gait Recognition , 2006, ACCV.

[38]  Alexandros Iosifidis,et al.  Movement recognition exploiting multi-view information , 2010, 2010 IEEE International Workshop on Multimedia Signal Processing.

[39]  A. Hilton,et al.  The i 3 DPost multiview and 3 D human action / interaction database , 2009 .

[40]  Ioannis Pitas,et al.  3D Human Action Recognition for Multi-view Camera Systems , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.