Practical Automated Video Analytics for Crowd Monitoring and Counting

Video surveillance is gaining popularity in numerous applications, including facility management, traffic monitoring, crowd analysis, and urban security. Despite the increasing demand for closed-circuit television (CCTV) and related infrastructure in public spaces, there remains a notable lack of readily-deployable automated surveillance systems. In this study, we present a low-cost and efficient approach that integrates the use of computational object recognition to perform fully-automated identification, tracking, and counting of human traffic on camera video streams. Two software implementations are explored and the performance of these schemes is compared. Validation against controlled and non-controlled real-world environments is also demonstrated. The implementation provides automated video analytics for medium crowd density monitoring and tracking, eliminating labor-intensive tasks traditionally requiring human operation, with results indicating great reliability in real-life scenarios.

[1]  Soraia Raupp Musse,et al.  Crowd Analysis Using Computer Vision Techniques , 2010, IEEE Signal Processing Magazine.

[2]  Mubarak Shah,et al.  Tracking and Object Classification for Automated Surveillance , 2002, ECCV.

[3]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[4]  Lu Zhang,et al.  Crowd Counting via Scale-Adaptive Convolutional Neural Network , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Robert Hudec,et al.  Comparison of Background Subtraction Methods on Near Infra-Red Spectrum Video Sequences☆ , 2017 .

[6]  David S. Ebert,et al.  Mobile analytics for emergency response and training , 2008 .

[7]  Yanjie Li,et al.  Face recognition based on convolutional neural network and support vector machine , 2016, 2016 IEEE International Conference on Information and Automation (ICIA).

[8]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[9]  Khalid Saeed,et al.  Moving Object Detection Using Background Subtraction , 2014, SpringerBriefs in Computer Science.

[10]  Anthony C. Caputo,et al.  Digital Video Surveillance and Security , 2010 .

[11]  Steven C. H. Hoi,et al.  Face Detection using Deep Learning: An Improved Faster RCNN Approach , 2017, Neurocomputing.

[12]  Trista Pei-chun Chen,et al.  Computer Vision Workload Analysis: Case Study of Video Surveillance Systems , 2005 .

[13]  Andrew A. Adams,et al.  The future of video analytics for surveillance and its ethical implications , 2015 .

[14]  Kari Pulli,et al.  Real-time computer vision with OpenCV , 2012, Commun. ACM.

[15]  Huaizu Jiang,et al.  Face Detection with the Faster R-CNN , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[16]  Christoph Stiller,et al.  The Role of Machine Vision for Intelligent Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[17]  Matthias Bethge,et al.  Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet , 2019, ICLR.

[18]  Haidi Ibrahim,et al.  Recent survey on crowd density estimation and counting for visual surveillance , 2015, Eng. Appl. Artif. Intell..

[19]  Dariusz Frejlichowski,et al.  Intelligent video surveillance systems for public spaces – a survey , 2014 .

[20]  A. Hampapur,et al.  Smart video surveillance: exploring the concept of multiscale spatiotemporal tracking , 2005, IEEE Signal Processing Magazine.

[21]  Ali Javed,et al.  Shot Classification of Field Sports Videos Using AlexNet Convolutional Neural Network , 2019, Applied Sciences.

[22]  Mario Hirz,et al.  Sensor and object recognition technologies for self-driving cars , 2017 .

[23]  Artem Korobov,et al.  Model and Training Methods of Autonomous Navigation System for Compact Drones , 2018, 2018 IEEE Second International Conference on Data Stream Mining & Processing (DSMP).

[24]  前田 俊二,et al.  Single Shot MultiBox DetectorとOptical Flowを組み合わせた逆走車両検知手法の検討 , 2018 .

[25]  Sergio A. Velastin.  CCTV Video Analytics: Recent Advances and Limitations , 2009, IVIC.

[26]  Varaprasad Golla,et al.  Implementing Intelligent Traffic Control System for Congestion Control, Ambulance Clearance, and Stolen Vehicle Detection , 2015, IEEE Sensors Journal.

[27]  Andrea Cavallaro,et al.  Video Analytics for Surveillance: Theory and Practice [From the Guest Editors] , 2010 .

[28]  Pascal Perez,et al.  Edge-Computing Video Analytics for Real-Time Traffic Monitoring in a Smart City , 2019, Sensors.

[29]  M. A. Sasse,et al.  Man or a Gorilla? Performance Issues with CCTV Technology in Security Control Rooms , 2006 .

[30]  Heng Wang,et al.  Robotics and Autonomous Systems , 2022 .

[31]  Mario Cifrek,et al.  A brief introduction to OpenCV , 2012, 2012 Proceedings of the 35th International Convention MIPRO.

[32]  Teddy Ko,et al.  A survey on behavior analysis in video surveillance for homeland security applications , 2008, 2008 37th IEEE Applied Imagery Pattern Recognition Workshop.

[33]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Seda Kul,et al.  Distributed and collaborative real-time vehicle detection and classification over the video streams , 2017 .

[35]  Bin Hui,et al.  Pedestrian Flow Tracking and Statistics of Monocular Camera Based on Convolutional Neural Network and Kalman Filter , 2019, Applied Sciences.

[36]  R. Lucas,et al.  Rule-based classification of multi-temporal satellite imagery for habitat and agricultural land cover mapping , 2007 .

[37]  Vahid Abrishami,et al.  Real-time pedestrian detecting and tracking in crowded and complicated scenario , 2009, ICDP.

[38]  N. Paragios,et al.  Video-Based Surveillance Systems: Computer Vision and Distributed Processing , 2001 .

[39]  Ahmet Sayar,et al.  SERVICE ORIENTED VISUAL INTERPRETATI ON TOOL FOR TIME SERIES DATA , 2013 .

[40]  Yang Li,et al.  Intelligent Video Surveillance System Based on Moving Object Detection and Tracking , 2016 .

[41]  Honghai Liu,et al.  Intelligent Video Systems and Analytics: A Survey , 2013, IEEE Transactions on Industrial Informatics.

[42]  David S. Ebert,et al.  Visual Analytics on Mobile Devices for Emergency Response , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[43]  Nadjia Benblidia,et al.  Comparison of Background Subtraction methods , 2018, 2018 International Conference on Applied Smart Systems (ICASS).

[44]  Ahmet Sayar,et al.  CNN Based Traffic Sign Recognition for Mini Autonomous Vehicles , 2018, Advances in Intelligent Systems and Computing.

[45]  P. KaewTrakulPong,et al.  An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection , 2002 .

[46]  Anton A. Zhilenkov,et al.  The use of convolution artificial neural networks for drones autonomous trajectory planning , 2018, 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus).

[47]  Kenneth Y. Goldberg,et al.  Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation , 2012, 2012 American Control Conference (ACC).

[48]  Stephen Balaban,et al.  Deep learning and face recognition: the state of the art , 2015, Defense + Security Symposium.

[49]  François Marmoiton,et al.  Toward Smart Autonomous Cars , 2016 .

[50]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[51]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[52]  Xu Kang,et al.  A Deep Similarity Metric Method Based on Incomplete Data for Traffic Anomaly Detection in IoT , 2019, Applied Sciences.

[53]  E. D. Dickmanns,et al.  The development of machine vision for road vehicles in the last decade , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[54]  Aftab Alam,et al.  SIAT: A Distributed Video Analytics Framework for Intelligent Video Surveillance , 2019, Symmetry.

[55]  David M. Brooks,et al.  Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective , 2018, 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[56]  Seda Kul,et al.  Performance Evaluation of Support Vector Machine and Convolutional Neural Network Algorithms in Real-Time Vehicle Type Classification , 2018, EIDWT.

[57]  Milind R. Naphade,et al.  Smarter Cities and Their Innovation Challenges , 2011, Computer.

[58]  P. Yakimov,et al.  CNN Design for Real-Time Traffic Sign Recognition , 2017 .

[59]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[60]  Seda Kul,et al.  Evaluation of Real-Time Performance for BGSLibrary Algorithms: A Case Study on Traffic Surveillance Video , 2016, 2016 6th International Conference on IT Convergence and Security (ICITCS).

[61]  Carlo S. Regazzoni,et al.  Bayesian Tracking for Video Analytics , 2010, IEEE Signal Processing Magazine.

[62]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[63]  Paramvir Bahl,et al.  Real-Time Video Analytics: The Killer App for Edge Computing , 2017, Computer.

[64]  Yong Wang,et al.  Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction , 2017, Sensors.

[65]  Laxmi Tyapi.  Real Time Human Detection from Video Surveillance , 2015 .