Depth Images-based Human Detection, Tracking and Activity Recognition Using Spatiotemporal Features and Modified HMM

Human activity recognition using depth information is an emerging and challenging technology in computer vision due to its considerable attention by many practical applications such as smart home/office system, personal health care and 3D video games. This paper presents a novel framework of 3D human body detection, tracking and recognition from depth video sequences using spatiotemporal features and modified HMM. To detect human silhouette, raw depth data is examined to extract human silhouette by considering spatial continuity and constraints of human motion information. While, frame differentiation is used to track human movements. Features extraction mechanism consists of spatial depth shape features and temporal joints features are used to improve classification performance. Both of these features are fused together to recognize different activities using the modified hidden Markov model (M-HMM). The proposed approach is evaluated on two challenging depth video datasets. Moreover, our system has significant abilities to handle subject"s body parts rotation and body parts missing which provide major contributions in human activity recognition.

[1]  Tae-Seong Kim,et al.  Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home , 2012, IEEE Transactions on Consumer Electronics.

[2]  Ahmad Jalal,et al.  Multiple Facial Feature Detection Using Vertex-Modeling Structure , 2007 .

[3]  Daijin Kim,et al.  Shape and Motion Features Approach for Activity Tracking and Recognition from Kinect Video Camera , 2015, 2015 IEEE 29th International Conference on Advanced Information Networking and Applications Workshops.

[4]  Ahmad Jalal,et al.  Global Security Using Human Face Understanding under Vision Ubiquitous Architecture System , 2008 .

[5]  Jake K. Aggarwal,et al.  Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Daijin Kim,et al.  A Depth Video Sensor-Based Life-Logging Human Activity Recognition System for Elderly Care in Smart Indoor Environments , 2014, Sensors.

[7]  Daijin Kim,et al.  Individual detection-tracking-recognition using depth activity images , 2015, 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[8]  Xiaodong Yang,et al.  EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[9]  Tae-Seong Kim,et al.  Human Activity Recognition via the Features of Labeled Depth Body Parts , 2012, ICOST.

[10]  Daijin Kim,et al.  Depth map-based human activity tracking and recognition using body joints features and Self-Organized Map , 2014, Fifth International Conference on Computing, Communications and Networking Technologies (ICCCNT).

[11]  Mian Ahmad Zeb,et al.  Security and QoS Optimization for Distributed Real Time Environment , 2007, 7th IEEE International Conference on Computer and Information Technology (CIT 2007).

[12]  Meinard Müller,et al.  Motion templates for automatic classification and retrieval of motion capture data , 2006, SCA '06.

[13]  Hans-Peter Seidel,et al.  A data-driven approach for real-time full body pose reconstruction from a depth camera , 2011, 2011 International Conference on Computer Vision.

[14]  Daijin Kim,et al.  Robust human activity recognition from depth video using spatiotemporal multi-fused features , 2017, Pattern Recognit..

[15]  A. Jalal,et al.  Security Architecture for Third Generation (3G) using GMHS Cellular Network , 2007, 2007 International Conference on Emerging Technologies.

[16]  Ahmad Jalal,et al.  The Mechanism of Edge Detection using the Block Matching Criteria for the Motion Estimation , 2005 .

[17]  Shaharyar Kamal,et al.  Dense RGB-D Map-Based Human Tracking and Activity Recognition using Skin Joints Features and Self-Organizing Map , 2015, KSII Trans. Internet Inf. Syst..

[18]  Ahmad Jalal,et al.  Advanced Performance Achievement using Multi- Algorithmic Approach of Video Transcoder for Low Bitrate Wireless Communication , 2005 .

[19]  Daijin Kim,et al.  Depth silhouettes context: A new robust feature for human tracking and activity recognition based on embedded HMMs , 2015, 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[20]  Tae-Seong Kim,et al.  Human Activity Recognition via Recognized Body Parts of Human Depth Silhouettes for Residents Monitoring Services at Smart Home , 2013 .

[21]  Daijin Kim,et al.  A spatiotemporal motion variation features extraction approach for human tracking and pose-based action recognition , 2015, 2015 International Conference on Informatics, Electronics & Vision (ICIEV).

[22]  Xu Sun,et al.  Large-Scale Personalized Human Activity Recognition Using Online Multitask Learning , 2013, IEEE Transactions on Knowledge and Data Engineering.

[23]  Ahmad Jalal,et al.  A Complexity Removal in the Floating Point and Rate Control Phenomenon , 2005 .

[24]  Ahmad Jalal,et al.  Dense depth maps-based human pose tracking and recognition in dynamic scenes using ridge data , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[25]  A. Jalal,et al.  Assembled algorithm in the real-time H.263 codec for advanced performance , 2005, Proceedings of 7th International Workshop on Enterprise networking and Computing in Healthcare Industry, 2005. HEALTHCOM 2005..

[26]  Xiaodong Yang,et al.  Super Normal Vector for Activity Recognition Using Depth Sequences , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Zicheng Liu,et al.  HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Shaharyar Kamal,et al.  A Hybrid Feature Extraction Approach for Human Detection, Tracking and Activity Recognition Using Depth Sensors , 2016 .

[29]  Tae-Seong Kim,et al.  Recognition of Human Home Activities via Depth Silhouettes and ℜ Transformation for Smart Homes , 2012 .

[30]  Daijin Kim,et al.  Human daily activity recognition with joints plus body features representation using Kinect sensor , 2015, 2015 International Conference on Informatics, Electronics & Vision (ICIEV).

[31]  Ying Wu,et al.  Mining actionlet ensemble for action recognition with depth cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Ahmad Jalal,et al.  Collaboration Achievement along with Performance Maintenance in Video Streaming , 2007 .

[33]  Shaharyar Kamal,et al.  Real-time life logging via a depth silhouette-based human activity recognition system for smart home services , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).