A Low-Dimensional Radial Silhouette-Based Feature for Fast Human Action Recognition Fusing Multiple Views

This paper presents a novel silhouette-based feature for vision-based human action recognition, which relies on the contour of the silhouette and a radial scheme. Its low-dimensionality and ease of extraction result in an outstanding proficiency for real-time scenarios. This feature is used in a learning algorithm that by means of model fusion of multiple camera streams builds a bag of key poses, which serves as a dictionary of known poses and allows converting the training sequences into sequences of key poses. These are used in order to perform action recognition by means of a sequence matching algorithm. Experimentation on three different datasets returns high and stable recognition rates. To the best of our knowledge, this paper presents the highest results so far on the MuHAVi-MAS dataset. Real-time suitability is given, since the method easily performs above video frequency. Therefore, the related requirements that applications as ambient-assisted living services impose are successfully fulfilled.

[1]  Du Tran,et al.  Human Activity Recognition with Metric Learning , 2008, ECCV.

[2]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[3]  Rémi Ronfard,et al.  Free viewpoint action recognition using motion history volumes , 2006, Comput. Vis. Image Underst..

[4]  Montse Pardàs,et al.  Human model and motion based 3D action recognition in multiple view scenarios , 2006, 2006 14th European Signal Processing Conference.

[5]  Mohan M. Trivedi,et al.  Human action recognition using multiple views: a comparative perspective on recent developments , 2011, J-HGBU '11.

[6]  Hsuan-Sheng Chen,et al.  Human action recognition using star skeleton , 2006, VSSN '06.

[7]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[8]  Miguel A. Patricio,et al.  A probabilistic, discriminative and distributed system for the recognition of human actions from multiple views , 2012, Neurocomputing.

[9]  Ioannis Pitas,et al.  The i3DPost Multi-View and 3D Human Action/Interaction Database , 2009, 2009 Conference for Visual Media Production.

[10]  Mubarak Shah,et al.  Learning 4D action feature models for arbitrary view action recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Zaw Zaw Htike,et al.  Model-free Viewpoint Invariant Human Activity Recognition , 2011 .

[12]  Hamid K. Aghajan,et al.  On efficient use of multi-view data for activity recognition , 2010, ICDSC '10.

[13]  Ramakant Nevatia,et al.  Single View Human Action Recognition using Key Pose Matching and Viterbi Path Searching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Vassilis G. Kaburlasos,et al.  A Lattice-Computing ensemble for reasoning based on formal fusion of disparate data types, and an industrial dispensing application , 2014, Inf. Fusion.

[15]  Jean-Christophe Nebel,et al.  Are Current Monocular Computer Vision Systems for Human Action Recognition Suitable for Visual Surveillance Applications? , 2011, ISVC.

[16]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[17]  Sergio A. Velastin,et al.  Recognizing Human Actions Using Silhouette-based HMM , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[18]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[19]  V. Ramasubramanian,et al.  Towards fast, view-invariant human action recognition , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Ying Wang,et al.  Human Activity Recognition Based on R Transform , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Ling Shao,et al.  Multi-view action recognition using local similarity random forests and sensor fusion , 2013, Pattern Recognit. Lett..

[22]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[23]  Q. M. Jonathan Wu,et al.  Incremental Learning in Human Action Recognition Based on Snippets , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[25]  KimKyungnam,et al.  Real-time foreground-background segmentation using codebook model , 2005 .

[26]  Ickjai Lee,et al.  Expert Systems With Applications , 2013 .

[27]  Miguel A. Patricio,et al.  Human action recognition with sparse classification and multiple‐view learning , 2014, Expert Syst. J. Knowl. Eng..

[28]  Alexandros André Chaaraoui,et al.  An Efficient Approach for Multi-view Human Action Recognition Based on Bag-of-Key-Poses , 2012, HBU.

[29]  Jonathan H. Connell,et al.  A Statistical Approach for Real-time Robust Background Subtrac tion and Shadow Detection , 2014 .

[30]  Juan José Pantrigo,et al.  Human Action Recognition Based on Tracking Features , 2011, IWINAC.

[31]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Pinar Duygulu Sahin,et al.  Human Action Recognition Using Distribution of Oriented Rectangular Patches , 2007, Workshop on Human Motion.

[33]  Pascal Fua,et al.  Making Action Recognition Robust to Occlusions and Viewpoint Changes , 2010, ECCV.

[34]  Chaur-Heh Hsieh,et al.  Human action recognition using silhouette histogram , 2011 .

[35]  Ayoub Al-Hamadi,et al.  A Fast Statistical Approach for Human Activity Recognition , 2012 .

[36]  Nicolas Pérez de la Blanca,et al.  HMM-Based Action Recognition Using Contour Histograms , 2007, IbPRIA.

[37]  Vassilis G. Kaburlasos,et al.  Binary Image 2D Shape Learning and Recognition Based on Lattice-Computing (LC) Techniques , 2011, Journal of Mathematical Imaging and Vision.

[38]  Hossein Ragheb,et al.  MuHAVi: A Multicamera Human Action Video Dataset for the Evaluation of Action Recognition Methods , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[39]  Chen Wu,et al.  Multiview activity recognition in smart homes with spatio-temporal features , 2010, ICDSC '10.

[40]  Christian Bauckhage,et al.  Action recognition by learning discriminative key poses , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[41]  Christian Bauckhage,et al.  Temporal key poses for human action recognition , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[42]  Alexandros André Chaaraoui,et al.  Silhouette-based human action recognition using sequences of key poses , 2013, Pattern Recognit. Lett..

[43]  Ronen Basri,et al.  Actions as space-time shapes , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[44]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[45]  Dimitrios Hatzinakos,et al.  Gait recognition using linear time normalization , 2006, Pattern Recognit..

[46]  Greg Mori,et al.  Action recognition by learning mid-level motion features , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[48]  José Manuel Ferrández,et al.  Foundations on Natural and Artificial Computation , 2011, Lecture Notes in Computer Science.

[49]  A. Enis Çetin,et al.  Silhouette-Based Method for Object Classification and Human Action Recognition in Video , 2006, ECCV Workshop on HCI.

[50]  Václav Hlavác,et al.  n -Grams of Action Primitives for Recognizing Human Behavior , 2007, CAIP.