Recognising human actions by analysing negative spaces

The authors propose a novel region-based method to recognise human actions. Other region-based approaches work on silhouette of the human body, which is termed as the positive space according to art theory. In contrast, the authors investigate and analyse regions surrounding the human body, termed as the negative space for human action recognition. This concept takes advantage of the naturally formed negative regions that come with simple shape, simplifying the job for action classification. Negative space is less sensitive to segmentation errors, overcoming some limitations of silhouette-based methods such as leaks or holes in the silhouette caused by background segmentation. Inexpensive semantic-level description can be generated from the negative space that supports fast and accurate action recognition. The proposed system has obtained 100% accuracy on the Weizmann human action dataset and the robust sequence dataset. On KTH dataset the system achieved 94.67% accuracy. Furthermore, 95% accuracy can be achieved even when half of the negative space regions are ignored. This makes our work robust with respect to segmentation errors and distinctive from other approaches.

[1]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[2]  M. Teague Image analysis via the general theory of moments , 1980 .

[3]  Jia-Guu Leu,et al.  Shape normalization through compacting , 1989, Pattern Recognit. Lett..

[4]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[5]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[6]  Gang Xu,et al.  Understanding human motion patterns , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[7]  Yee-Hong Yang,et al.  First Sight: A Human Body Outline Labeling System , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[11]  Jake K. Aggarwal,et al.  Segmentation and recognition of continuous human activity , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[12]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[14]  Ivan Laptev,et al.  On Space-Time Interest Points , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  Yonghuai Liu,et al.  Improving ICP with easy implementation for free-form surface matching , 2004, Pattern Recognit..

[16]  Daniel P. Huttenlocher,et al.  Pictorial Structures for Object Recognition , 2004, International Journal of Computer Vision.

[17]  D. Hatzinakos,et al.  Gait recognition: a challenging signal processing technology for biometric identification , 2005, IEEE Signal Processing Magazine.

[18]  Roman Goldenberg,et al.  Behavior classification by eigendecomposition of periodic motions , 2005, Pattern Recognit..

[19]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[20]  Bruno Raffin,et al.  3D Skeleton-Based Body Pose Recovery , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[21]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[22]  Ankur Agarwal,et al.  Recovering 3D human pose from monocular images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[24]  Andrew Zisserman,et al.  Tracking People by Learning Their Appearance , 2007 .

[25]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words , 2008, International Journal of Computer Vision.

[26]  Thomas Serre,et al.  A Biologically Inspired System for Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[27]  Ronen Basri,et al.  Actions as Space-Time Shapes , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Mubarak Shah,et al.  A 3-dimensional sift descriptor and its application to action recognition , 2007, ACM Multimedia.

[29]  Wanqing Li,et al.  Graphical modeling and decoding of human actions , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[30]  Qiang Wu,et al.  Human Action Recognition by Radon Transform , 2008, 2008 IEEE International Conference on Data Mining Workshops.

[31]  Andrew Zisserman,et al.  2D Human Pose Estimation in TV Shows , 2009, Statistical and Geometrical Approaches to Visual Motion Analysis.

[32]  Mohiuddin Ahmad,et al.  Human action recognition using shape and CLG-motion flow from multi-view image sequences , 2008, Pattern Recognit..

[33]  David A. Forsyth,et al.  Searching for Complex Human Activities with No Visual Examples , 2008, International Journal of Computer Vision.

[34]  W. Eric L. Grimson,et al.  Unsupervised Activity Perception in Crowded and Complicated Scenes Using Hierarchical Bayesian Models , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Pinar Duygulu Sahin,et al.  Histogram of oriented rectangles: A new pose descriptor for human action recognition , 2009, Image Vis. Comput..

[36]  Jake K. Aggarwal,et al.  Human action recognition with extremities as semantic posture representation , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[37]  Ramakant Nevatia,et al.  Human Pose Tracking in Monocular Sequence Using Multilevel Structured Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Michael Hofmann,et al.  Single-Frame 3D Human Pose Recovery from Multiple Views , 2009, DAGM-Symposium.

[39]  Manuel J. Marín-Jiménez,et al.  Human Action Recognition Using Optical Flow Accumulated Local Histograms , 2009, IbPRIA.

[40]  Wei Liang,et al.  Incremental discriminant-analysis of canonical correlations for action recognition , 2010, Pattern Recognit..

[41]  Wei Xiong,et al.  Active energy image plus 2DLPP for gait recognition , 2010, Signal Process..

[42]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[43]  Boubakeur Boufama,et al.  A Novel Human Motion Recognition Method Based on Eigenspace , 2010, ICIAR.

[44]  Peyman Milanfar,et al.  Action Recognition from One Example , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Ling Shao,et al.  Transform based spatio-temporal descriptors for human action recognition , 2011, Neurocomputing.

[46]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[47]  James Nga-Kwok Liu,et al.  Gait flow image: A silhouette-based gait representation for human identification , 2011, Pattern Recognit..