Analysis of Motion Self-Occlusion Problem Due to Motion Overwriting for Human Activity Recognition

Various recognition methodologies address to recognize and understand varieties of human activities. However, motion self-occlusion due to motion overlapping in the same region is a daunting task to solve. Various motion-recognition methods either bypass this problem or solve this problem in complex manner. Appearance-based template matching paradigms are simpler and hence these approaches faster for activity analysis. In this paper, we concentrate on motion self- occlusion problem due to motion overlapping in various complex activities for recognition. In the Motion History Image (MHI) method, the self-occlusion is evident and it should be solved. Therefore, this paper compares our directional motion history image concept with basic the Motion History Image, Multi-level Motion History representation and Hierarchical Motion History Histogram representation to solve the self-occlusion problem of basic the Motion History Image representation. We employ some complex aerobics and find the robustness of our method compared to other methods for this self-occlusion problem. We employ seven higher order Hu moments to compute the feature vector for each activity. Afterwards, k-nearest neighbor method is utilized for classification with leave-one-out paradigm. The comparative results clearly demonstrate the superiority of our method than other recent approaches. We also present several experiments to demonstrate the performance and strength of the DMHI method in recognizing various complex actions. Index Terms—MHI, DMHI, MMHI, HMHH, motion recognition, feature vector

[1]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[2]  Morshed U. Chowdhury,et al.  Image semantic classification by using SVM , 2003 .

[3]  Xiongfei Li,et al.  Research and Design of Drawings Retrieval System Based on Spatial Relationships and Shape , 2008, 2008 ISECS International Colloquium on Computing, Communication, Control, and Management.

[4]  Pierre Comon Independent component analysis - a new concept? signal processing , 1994 .

[5]  Xiongfei Li,et al.  Adaptive Feature Selection and Extraction Approaches for Image Retrieval based on Region , 2010, J. Multim..

[6]  Hans-Peter Seidel,et al.  3D scattered data approximation with adaptive compactly supported radial basis functions , 2004, Proceedings Shape Modeling Applications, 2004..

[7]  Chengjun Liu,et al.  Gabor-based kernel PCA with fractional power polynomial models for face recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Michel Barlaud,et al.  Image coding using wavelet transform , 1992, IEEE Trans. Image Process..

[9]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  M. Ahmad,et al.  A combined local-global optical flow approach for cranial ultrasonogram image sequence analysis , 2008, 2008 11th International Conference on Computer and Information Technology.

[11]  Maria Grazia Albanesi,et al.  Wavelets and human visual perception in image compression , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[12]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2005, International Journal of Computer Vision.

[13]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Colin Fyfe,et al.  Kernel and Nonlinear Canonical Correlation Analysis , 2000, IJCNN.

[15]  S. Ishikawa,et al.  Action recognition with various speeds and timed-DMHI feature vectors , 2008, 2008 11th International Conference on Computer and Information Technology.

[16]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[17]  Nicu Sebe,et al.  How to complete performance graphs in content-based image retrieval: add generality and normalize scope , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  R A Older Using the photo CD in academic radiology. , 1996, AJR. American journal of roentgenology.

[19]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[20]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[21]  Joo Kooi Tan,et al.  High-Speed Human Motion Recognition Based on a Motion History Image and an Eigenspace , 2006, IEICE Trans. Inf. Syst..

[22]  Leif Kobbelt,et al.  A survey of point-based techniques in computer graphics , 2004, Comput. Graph..

[23]  R. Fabio From point cloud to surface the modeling and visualization problem , 2003 .

[24]  Shotaro Akaho,et al.  A kernel method for canonical correlation analysis , 2006, ArXiv.

[25]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Kalpathi R. Subramanian,et al.  Interpolating implicit surfaces from scattered surface data using compactly supported radial basis functions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[27]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[28]  W. S. Warner Recent Developments in the USDA’s 35 mm Aerial Photography Programme , 1996 .

[29]  Marc Alexa,et al.  Reconstruction with Voronoi centered radial basis functions , 2006, SGP '06.

[30]  David E. Booth,et al.  Applied Multivariate Analysis , 2003, Technometrics.

[31]  Do Lenh Hung Son,et al.  Detection and localization of road area in traffic video sequences using motion information and fuzzy-shadowed sets , 2005, Seventh IEEE International Symposium on Multimedia (ISM'05).

[32]  Karianto Leman,et al.  Pda Based Human Motion Recognition System , 2005, Int. J. Softw. Eng. Knowl. Eng..

[33]  Robert T. Collins,et al.  Moving Object Localization in Thermal Imagery by Forward-backward MHI , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[34]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  M. Pantic,et al.  Facial action unit recognition using temporal templates , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[37]  Loris Nanni,et al.  Weighted Sub-Gabor for face recognition , 2007, Pattern Recognit. Lett..

[38]  Alex Pentland,et al.  Human computing and machine understanding of human behavior: a survey , 2006, ICMI '06.

[39]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[40]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[41]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[42]  Nanning Zheng,et al.  Gait History Image: A Novel Temporal Template for Gait Recognition , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[43]  S. Ishikawa,et al.  Human activity recognition: Various paradigms , 2008, 2008 International Conference on Control, Automation and Systems.

[44]  William J. Christmas Spatial Filtering Requirements for Gradient-Based Optical Flow Measurement , 1998, BMVC.

[45]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[46]  Wen Liu Accuracy and reliability of a technique for quantifying foot shape, dimensions and structural characteristics , 1999 .

[47]  Rama Chellappa,et al.  View invariants for human action recognition , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[48]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[49]  Jorge Núñez,et al.  Super-Resolution of Remotely Sensed Images With Variable-Pixel Linear Reconstruction , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Ramakant Nevatia,et al.  Tracking of Multiple, Partially Occluded Humans based on Static Body Part Detection , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[51]  Giovanni Ramponi,et al.  A simple edge-sensitive image interpolation filter , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[52]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[53]  Martial Hebert,et al.  Efficient visual event detection using volumetric features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[54]  José Carlos M. Bermudez,et al.  Statistical Analysis of the LMS Algorithm Applied to Super-Resolution Image Reconstruction , 2006, IEEE Transactions on Signal Processing.

[55]  Maja Pantic,et al.  Motion history for facial action detection in video , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[56]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[57]  Hans-Peter Seidel,et al.  Multi-level partition of unity implicits , 2005, SIGGRAPH Courses.

[58]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[59]  Seong-Won Lee,et al.  Image interpolation using adaptive fast B-spline filtering , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[60]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[61]  Mei-Chen Yeh,et al.  Fast Human Detection Using a Cascade of Histograms of Oriented Gradients , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[62]  Ronald Poppe,et al.  Vision-based human motion analysis: An overview , 2007, Comput. Vis. Image Underst..

[63]  Matti Pietikäinen,et al.  Texture Based Description of Movements for Activity Analysis , 2008, VISAPP.

[64]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[65]  Mostafa Zaman Chowdhury,et al.  GLOBAL OPTIMIZATION OF OPTICAL FLOW TECHNIQUE ON THE COMPUTATION OF TISSUE-MOTION IN NEONATAL CRANIAL ULTRASONOGRAM , 2004 .

[66]  Sanjay Kumar,et al.  Visual Speech Recognition Using Image Moments and Multiresolution Wavelet Images , 2006, International Conference on Computer Graphics, Imaging and Visualisation (CGIV'06).

[67]  Kenneth E. Barner,et al.  A Computationally Efficient Super-Resolution Algorithm for Video Processing Using Partition Filters , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[68]  James W. Davis,et al.  Virtual PAT: A Virtual Personal Aerobics Trainer , 1998 .

[69]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[70]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[71]  J. Schmee An Introduction to Multivariate Statistical Analysis , 1986 .

[72]  Md. Atiqur Rahman Ahad,et al.  Motion recognition approach to solve overwriting in complex actions , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[73]  S. Ishikawa,et al.  Directional motion history templates for low resolution motion recognition , 2008, 2008 34th Annual Conference of IEEE Industrial Electronics.

[74]  S. Ishikawa,et al.  Solutions to motion self-occlusion problem in human activity analysis , 2008, 2008 11th International Conference on Computer and Information Technology.

[75]  Ze-Nian Li,et al.  Successive Convex Matching for Action Detection , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[76]  Guojun Lu,et al.  Review of shape representation and description techniques , 2004, Pattern Recognit..

[77]  J. Sullivan,et al.  Action Recognition by Shape Matching to Key Frames , 2002 .

[78]  James F. O'Brien,et al.  Shape transformation using variational implicit functions , 1999, SIGGRAPH Courses.

[79]  István Petrás,et al.  Flexible test-bed for unusual behavior detection , 2007, CIVR '07.

[80]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[81]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[82]  Chih-Jen Lin,et al.  Simple Probabilistic Predictions for Support Vector Regression , 2004 .

[83]  Ramakant Nevatia,et al.  Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[84]  H. Liedholm,et al.  The development of an interactive education program for heart failure patients: the Kodak Photo CD Portfolio concept. , 1996, Patient education and counseling.

[85]  Masayuki Fukuzawa,et al.  Motion analysis of artery pulsation in neonatal cranial ultrasonogram , 1999, Medical Imaging.

[86]  Masayuki Fukuzawa,et al.  Pulsation Detection from Noisy Ultrasound-Echo Moving Images of Newborn Baby Head Using Fourier Transform , 1995 .

[87]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[88]  Zi-Quan Hong,et al.  Algebraic feature extraction of image for recognition , 1991, Pattern Recognit..

[89]  Richard M. Leahy,et al.  An Optimal Graph Theoretic Approach to Data Clustering: Theory and Its Application to Image Segmentation , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  In-Kwon Lee,et al.  Curve reconstruction from unorganized points , 2000, Comput. Aided Geom. Des..

[91]  Sabine Süsstrunk,et al.  A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution , 2006, EURASIP J. Adv. Signal Process..

[92]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[93]  D. Boone,et al.  Quantitative comparison of plantar foot shapes under different weight-bearing conditions. , 2003, Journal of rehabilitation research and development.

[94]  Oscar Déniz-Suárez,et al.  Face recognition using independent component analysis and support vector machines , 2001, Pattern Recognit. Lett..

[95]  Gary R. Bradski,et al.  Motion segmentation and pose recognition with motion history gradients , 2002, Machine Vision and Applications.

[96]  S. Osher,et al.  Fast surface reconstruction using the level set method , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[97]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[98]  Hongzhou Zhang,et al.  A Novel 2D Gabor Wavelets Window Method for Face Recognition , 2006, MRCS.

[99]  Martin D. Levine,et al.  Face Recognition Using the Discrete Cosine Transform , 2001, International Journal of Computer Vision.

[100]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[101]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[102]  Wei Lu,et al.  4D-CT motion estimation using deformable image registration and 5D respiratory motion modeling. , 2008, Medical physics.

[103]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[104]  Ming Jiang,et al.  A Surface Reconstruction Method for Highly Noisy Point Clouds , 2005, VLSM.

[105]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words , 2006, BMVC.

[106]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[107]  Hongying Meng,et al.  A Human Action Recognition System for Embedded Computer Vision Application , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[108]  Tomaso A. Poggio,et al.  Pedestrian detection using wavelet templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[109]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[110]  Greg Mori,et al.  Detecting Pedestrians by Learning Shapelet Features , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[111]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[112]  Richard K. Beatson,et al.  Surface interpolation with radial basis functions for medical imaging , 1997, IEEE Transactions on Medical Imaging.

[113]  J. Weickert,et al.  Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods , 2005 .

[114]  A. Laub,et al.  The singular value decomposition: Its computation and some applications , 1980 .

[115]  Dariu Gavrila,et al.  An Experimental Study on Pedestrian Classification , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[116]  Osama Masoud,et al.  Recognizing human activities , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[117]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[118]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[119]  José Carlos Príncipe,et al.  Super-resolution of images based on local correlations , 1999, IEEE Trans. Neural Networks.

[120]  T. Jan,et al.  Neural network based threat assessment for automated visual surveillance , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[121]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[122]  Masayuki Fukuzawa,et al.  A Real-Time Processing System for Pulsation Detection in Neonatal Cranial Ultrasonogram , 1998 .

[123]  Dariu Gavrila,et al.  Pedestrian Detection from a Moving Vehicle , 2000, ECCV.

[124]  Gary Bradski,et al.  Real-time Motion Template Gradients using Intel CVLib , 1999 .

[125]  Ravindra S. Goonetilleke,et al.  Foot measurements from three-dimensional scans: A comparison and evaluation of different methods , 2006 .

[126]  Hans-Peter Seidel,et al.  3D scattered data interpolation and approximation with multilevel compactly supported RBFs , 2005, Graph. Model..

[127]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[128]  Hongying Meng,et al.  Recognizing heuman actions based on motion information and SVM , 2006 .

[129]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[130]  Nikolas P. Galatsanos,et al.  Super-Resolution Based on Fast Registration and Maximum a Posteriori Reconstruction , 2007, IEEE Transactions on Image Processing.

[131]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[132]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[133]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[134]  Narendra Ahuja,et al.  POCS based adaptive image magnification , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[135]  Ramakant Nevatia,et al.  Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors , 2007, International Journal of Computer Vision.

[136]  Li Li,et al.  Spatio-temporal motion segmentation and tracking under realistic condition , 2006, 2006 1st International Symposium on Systems and Control in Aerospace and Astronautics.

[137]  Yaser Sheikh,et al.  Exploring the space of a human action , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[138]  A Wolfenden,et al.  Foot Measurement Strategies for Fitting Athletes , 1988 .

[139]  Hongying Meng,et al.  Motion information combination for fast human action recognition , 2007, VISAPP.

[140]  Gary E. Ford,et al.  Directional interpolation of images based on visual properties and rank order filtering , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[141]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[142]  Ahmed M. Darwish,et al.  Adaptive resampling algorithm for image zooming , 1996, Electronic Imaging.