Multi-Person Tracking in Smart Surveillance System for Crowd Counting and Normal/Abnormal Events Detection

Automated video surveillance addresses people's real-time observation to describe their behaviors and interactions. This paper presents a novel multi-person tracking system for crowd counting and normal/ abnormal events detection at indoor/outdoor surveillance environments. The proposed system consists of four modules: people detection, head-torso template extraction, tracking and crowd cluster analysis. Firstly, the system extracts human silhouettes using inverse transform as well as median filter reducing the cost of computing and handling various complex monitoring situations. Secondly, people are detected by their head torso due to less varied and hardly occluded. Thirdly, each person is tracked through consecutive frames using the Kalman filter techniques with Jaccard similarity and normalized cross-correlation. Finally, the template marking is used for crowd counting having cues localization and clustered via Gaussian mapping for normal/abnormal events detection. The experimental results on two challenging datasets of video surveillance such as PETS2009 and UMN crowd analysis datasets demonstrate that the proposed system provides 88.7% and 95.5% in terms of counting accuracy and detection rate.

[1]  Serge J. Belongie,et al.  Counting Crowded Moving Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  A. Jalal,et al.  Security Architecture for Third Generation (3G) using GMHS Cellular Network , 2007, 2007 International Conference on Emerging Technologies.

[3]  Daijin Kim,et al.  A Depth Video-based Human Detection and Activity Recognition using Multi-features and Embedded Hidden Markov Models for Health Care Monitoring Systems , 2017, Int. J. Interact. Multim. Artif. Intell..

[4]  K. Fazekas,et al.  Threshold procedures and image segmentation , 2005, 47th International Symposium ELMAR, 2005..

[5]  Joris De Schutter,et al.  An adaptable system for RGB-D based human body detection and pose estimation , 2014, J. Vis. Commun. Image Represent..

[6]  Maria Mahmood,et al.  Students’ behavior mining in e-learning environment using cognitive processes with information technologies , 2019, Education and Information Technologies.

[7]  Ahmad Jalal,et al.  A Complexity Removal in the Floating Point and Rate Control Phenomenon , 2005 .

[8]  Shaharyar Kamal,et al.  Dense RGB-D Map-Based Human Tracking and Activity Recognition using Skin Joints Features and Self-Organizing Map , 2015, KSII Trans. Internet Inf. Syst..

[9]  Yun Yuan,et al.  Real-Time Human Behavior Recognition in Intelligent Environment , 2006, 2006 8th international Conference on Signal Processing.

[10]  Tsong-Yi Chen,et al.  An Intelligent People-Flow Counting Method for Passing Through a Gate , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[11]  Ahmad Jalal,et al.  Facial Expression Recognition in Image Sequences Using 1D Transform and Gabor Wavelet Transform , 2018, 2018 International Conference on Applied and Engineering Mathematics (ICAEM).

[12]  Yannick Boursier,et al.  Sparsity-driven people localization algorithm: Evaluation in crowded scenes environments , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[13]  Ahmad Jalal,et al.  Advanced Performance Achievement using Multi- Algorithmic Approach of Video Transcoder for Low Bitrate Wireless Communication , 2005 .

[14]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[15]  Sung-Jea Ko,et al.  Real-time Vision-based People Counting System for the Security Door , 2002 .

[16]  Daijin Kim,et al.  Human daily activity recognition with joints plus body features representation using Kinect sensor , 2015, 2015 International Conference on Informatics, Electronics & Vision (ICIEV).

[17]  Ahmad Jalal,et al.  Wearable Sensor-Based Human Behavior Understanding and Recognition in Daily Life for Smart Environments , 2018, 2018 International Conference on Frontiers of Information Technology (FIT).

[18]  Atta Badii,et al.  Robust background model for pixel based people counting using a single uncalibrated camera , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[19]  Tao Ding,et al.  Fast track matching and event detection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Tullio Vernazza,et al.  Analysis of human behavior recognition algorithms based on acceleration data , 2013, 2013 IEEE International Conference on Robotics and Automation.

[21]  Ahmad Jalal,et al.  Robust Spatio-Temporal Features for Human Interaction Recognition Via Artificial Neural Network , 2018, 2018 International Conference on Frontiers of Information Technology (FIT).

[22]  Daijin Kim,et al.  Robust human activity recognition from depth video using spatiotemporal multi-fused features , 2017, Pattern Recognit..

[23]  Damla Arifoglu,et al.  Activity Recognition and Abnormal Behaviour Detection with Recurrent Neural Networks , 2017, FNC/MobiSPC.

[24]  Shaharyar Kamal,et al.  Improved Behavior Monitoring and Classification Using Cues Parameters Extraction from Camera Array Images , 2019, Int. J. Interact. Multim. Artif. Intell..

[25]  J. Ferryman,et al.  PETS2009: Dataset and challenge , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[26]  Daijin Kim,et al.  Individual detection-tracking-recognition using depth activity images , 2015, 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[27]  Maria Mahmood,et al.  Multi-features descriptors for Human Activity Tracking and Recognition in Indoor-Outdoor Environments , 2019, 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST).

[28]  Tae-Seong Kim,et al.  Human Activity Recognition via Recognized Body Parts of Human Depth Silhouettes for Residents Monitoring Services at Smart Home , 2013 .

[29]  Daijin Kim,et al.  Depth map-based human activity tracking and recognition using body joints features and Self-Organized Map , 2014, Fifth International Conference on Computing, Communications and Networking Technologies (ICCCNT).

[30]  Ahmad Jalal,et al.  The Mechanism of Edge Detection using the Block Matching Criteria for the Motion Estimation , 2005 .

[31]  Paul L. Rosin,et al.  Machine Vision and Applications Detecting Violent and Abnormal Crowd Activity Using Temporal Analysis of Grey Level Co-occurrence Matrix (glcm)-based Texture Measures , 2022 .

[32]  Shaharyar Kamal,et al.  Real-time life logging via a depth silhouette-based human activity recognition system for smart home services , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[33]  Lihong Zheng,et al.  Facial expression recognition using hybrid features and self-organizing maps , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[34]  Hamid R. Rabiee,et al.  Novel dataset for fine-grained abnormal behavior understanding in crowd , 2016, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[35]  Daijin Kim,et al.  Ridge body parts features for human pose estimation and recognition from RGB-D video data , 2014, Fifth International Conference on Computing, Communications and Networking Technologies (ICCCNT).

[36]  Xiaodong Yang,et al.  EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[37]  Dong-Seong Kim,et al.  Detecting Complex 3D Human Motions with Body Model Low-Rank Representation for Real-Time Smart Activity Monitoring System , 2018, KSII Trans. Internet Inf. Syst..

[38]  Daijin Kim,et al.  A spatiotemporal motion variation features extraction approach for human tracking and pose-based action recognition , 2015, 2015 International Conference on Informatics, Electronics & Vision (ICIEV).

[39]  Ahmad Jalal,et al.  A Triaxial Acceleration-based Human Motion Detection for Ambient Smart Home System , 2019, 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST).

[40]  Tae-Seong Kim,et al.  Human Activity Recognition via the Features of Labeled Depth Body Parts , 2012, ICOST.

[41]  Jean-Philippe Thiran,et al.  Counting Pedestrians in Video Sequences Using Trajectory Clustering , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Shaogang Gong,et al.  Tracking multiple people with a multi-camera system , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[43]  Shaharyar Kamal,et al.  A Hybrid Feature Extraction Approach for Human Detection, Tracking and Activity Recognition Using Depth Sensors , 2016 .

[44]  Ahmad Jalal,et al.  Dense depth maps-based human pose tracking and recognition in dynamic scenes using ridge data , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[45]  A. Jalal,et al.  Assembled algorithm in the real-time H.263 codec for advanced performance , 2005, Proceedings of 7th International Workshop on Enterprise networking and Computing in Healthcare Industry, 2005. HEALTHCOM 2005..

[46]  Sangwook Kim,et al.  Algorithmic implementation and efficiency maintenance of real-time environment using low-bitrate wireless communication , 2006, The Fourth IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems, and the Second International Workshop on Collaborative Computing, Integration, and Assurance (SEUS-WCCIA'06).

[47]  Nuno Vasconcelos,et al.  Analysis of Crowded Scenes using Holistic Properties , 2009 .

[48]  Tijo Thomas,et al.  Video summarization by clustering using euclidean distance , 2011, 2011 International Conference on Signal Processing, Communication, Computing and Networking Technologies.

[49]  Ahmad Jalal,et al.  Multiple Facial Feature Detection Using Vertex-Modeling Structure , 2007 .

[50]  Weihua Sheng,et al.  Wearable Sensor-Based Behavioral Anomaly Detection in Smart Assisted Living Systems , 2015, IEEE Transactions on Automation Science and Engineering.

[51]  Daijin Kim,et al.  Shape and Motion Features Approach for Activity Tracking and Recognition from Kinect Video Camera , 2015, 2015 IEEE 29th International Conference on Advanced Information Networking and Applications Workshops.

[52]  Daijin Kim,et al.  Depth silhouettes context: A new robust feature for human tracking and activity recognition based on embedded HMMs , 2015, 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[53]  Kibum Kim,et al.  A Wrist Worn Acceleration Based Human Motion Analysis and Classification for Ambient Smart Home System , 2019, Journal of Electrical Engineering & Technology.

[54]  Bala Srinivasan,et al.  Adaptive mobile activity recognition system with evolving data streams , 2015, Neurocomputing.

[55]  Cesar A. Azurdia-Meza,et al.  Depth Maps-Based Human Segmentation and Action Recognition Using Full-Body Plus Body Color Cues Via Recognizer Engine , 2019, Journal of Electrical Engineering & Technology.