Detection of stationary foreground objects: A survey

Survey of the most relevant approaches for detecting stationary foreground objects.Kinds of stationary foreground objects in the literature.Typical challenges to overcome, and popular datasets to test the strategies.Descriptions of the algorithms applied in the different stages of the strategies. Detection of stationary foreground objects (i.e., moving objects that remain static throughout several frames) has attracted the attention of many researchers over the last decades and, consequently, many new ideas have been recently proposed, trying to achieve high-quality detections in complex scenarios with the lowest misdetections, while keeping real-time constraints. Most of these strategies are focused on detecting abandoned objects. However, there are some approaches that also allow detecting partially-static foreground objects (e.g. people remaining temporarily static) or stolen objects (i.e., objects removed from the background of the scene).This paper provides a complete survey of the most relevant approaches for detecting all kind of stationary foreground objects. The aim of this survey is not to compare the existing methods, but to provide the information needed to get an idea of the state of the art in this field: kinds of stationary foreground objects, main challenges in the field, main datasets for testing the detection of stationary foreground, main stages in the existing approaches and algorithms typically used in such stages.

[1]  Philip Birch,et al.  Illumination invariant stationary object detection , 2013, IET Comput. Vis..

[2]  Adolfo López,et al.  Exploiting scene cues for dropped object detection , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[3]  Larry S. Davis,et al.  A Robust Background Subtraction and Shadow Detection , 1999 .

[4]  Hefeng Wu,et al.  Hierarchical Ensemble of Background Models for PTZ-Based Video Surveillance , 2015, IEEE Transactions on Cybernetics.

[5]  Quan Pan,et al.  Multiple layer based background maintenance in complex environment , 2004, Third International Conference on Image and Graphics (ICIG'04).

[6]  Bernt Schiele,et al.  Automatic Detection and Tracking of Abandoned Objects , 2003 .

[7]  Sergio A. Velastin,et al.  A real time surveillance system for metropolitan railways , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[8]  Yingli Tian,et al.  Real-Time Detection of Abandoned and Removed Objects in Complex Environments , 2008 .

[9]  Antoine Vacavant,et al.  A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos , 2014, Comput. Vis. Image Underst..

[10]  Wei Tsang Ooi,et al.  Detecting Static Objects in Busy Scenes , 1999 .

[11]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[12]  Xingzhi Luo,et al.  A Multiscale Parametric Background Model for Stationary Foreground Object Detection , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[13]  Fengjun Lv,et al.  Left-Luggage Detection using Bayesian Inference , 2006 .

[14]  Anupam Agrawal,et al.  Unusual activity detection for video surveillance , 2010, IITM '10.

[15]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[16]  Randal W. Beard,et al.  Multiple target tracking and stationary object detection in video with Recursive-RANSAC and tracker-sensor feedback , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[17]  Longin Jan Latecki,et al.  Object Tracking with Dynamic Template Update and Occlusion Detec , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[18]  José María Martínez Sanchez,et al.  Comparative Evaluation of Stationary Foreground Object Detection Algorithms Based on Background Subtraction Techniques , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[19]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[20]  Rubén Heras Evangelio,et al.  Static Object Detection Based on a Dual Background Model and a Finite-State Machine , 2011, EURASIP J. Image Video Process..

[21]  Marko Heikkilä,et al.  A texture-based method for modeling the background and detecting moving objects , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Hong Liu,et al.  Robust abandoned object detection and analysis based on online learning , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[23]  Alessandro Bevilacqua,et al.  Real time detection of stopped vehicles in traffic scenes , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[24]  Nilesh J. Uke,et al.  An Approach for Detecting Abandoned Object from Real Time Video , 2012 .

[25]  Zhijing Liu,et al.  An Evolutionary Approach to Image Noise Cancellation Using Adaptive Particle Swarm Optimization (APSO) , 2010, 2010 2nd International Conference on Computational Intelligence, Communication Systems and Networks.

[26]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[27]  Thierry Bouwmans,et al.  Traditional and recent approaches in background modeling for foreground detection: An overview , 2014, Comput. Sci. Rev..

[28]  Eduardo A. B. da Silva,et al.  On the detection of abandoned objects with a moving camera using robust subspace recovery and sparse representation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  Max Lu,et al.  Robust and efficient foreground analysis for real-time video surveillance , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Frédéric Guichard,et al.  Time-scale change detection applied to real-time abnormal stationarity monitoring , 2004, Real Time Imaging.

[31]  Kuo-Chin Fan,et al.  Carried Object Detection Using Ratio Histogram and its Application to Suspicious Event Analysis , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Anlong Ming,et al.  Abandoned object detection in highway scene , 2011, 2011 6th International Conference on Pervasive Computing and Applications.

[33]  Zhiquan Wang,et al.  Robust Real-Time Detection of Abandoned and Removed Objects , 2009, 2009 Fifth International Conference on Image and Graphics.

[34]  Nigel J. B. McFarlane,et al.  Segmentation and tracking of piglets in images , 1995, Machine Vision and Applications.

[35]  Kahlil Muchtar,et al.  Abandoned object detection in complicated environments , 2013, 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.

[36]  Amar Mitiche,et al.  Context-independent real-time event recognition: application to key-image extraction , 2002, Object recognition supported by user interaction for service robots.

[37]  Lucia Maddalena,et al.  Stopped Object Detection by Learning Foreground Model in Videos , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Tiziana D'Orazio,et al.  An Abandoned/Removed Objects Detection Algorithm and Its Evaluation on PETS Datasets , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[39]  Yunde Jia,et al.  Spatio-temporal patches for night background modeling by subspace learning , 2008, 2008 19th International Conference on Pattern Recognition.

[40]  Thierry Bouwmans,et al.  Subspace Learning for Background Modeling: A Survey , 2009 .

[41]  Brian C. Lovell,et al.  An Abandoned Object Detection System Based on Dual Background Segmentation , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[42]  Rubén Heras Evangelio,et al.  Complementary background models for the detection of static and moving objects in crowded environments , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[43]  Vittorio Murino,et al.  Structured neural networks for pattern recognition , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[44]  Thierry Bouwmans,et al.  Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey , 2008 .

[45]  Daijin Kim,et al.  Abnormal Object Detection Using Feedforward Model and Sequential Filters , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[46]  N. Papanikolopoulos,et al.  Practical mixtures of Gaussians with brightness monitoring , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[47]  Sharath Pankanti,et al.  Robust Foreground and Abandonment Analysis for Large-Scale Abandoned Object Detection in Complex Surveillance Videos , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[48]  Juan C. SanMiguel,et al.  Multi-feature stationary foreground detection for crowded video-surveillance , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[49]  Lei Geng,et al.  Real Time Foreground-Background Segmentation Using Two-Layer Codebook Model , 2011, 2011 International Conference on Control, Automation and Systems Engineering (CASE).

[50]  Dragoljub Pokrajac,et al.  Detecting and Recognizing Abandoned Objects in Crowded Environments , 2008, ICVS.

[51]  Sharath Pankanti,et al.  Modeling of temporarily static objects for robust abandoned object detection in urban surveillance , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[52]  D.L. Vilarino,et al.  CNN implementation of a moving object segmentation approach for real-time video surveillance , 2008, 2008 11th International Workshop on Cellular Neural Networks and Their Applications.

[53]  Simon Weidert,et al.  Machine learning-based augmented reality for improved surgical scene understanding , 2015, Comput. Medical Imaging Graph..

[54]  Suchendra M. Bhandarkar,et al.  Real-Time and Robust Background Updating for Video Surveillance and Monitoring , 2005, ICIAR.

[55]  Chien-Hung Liu,et al.  Intelligent Detection of Missing and Unattended Objects in Complex Scene of Surveillance Videos , 2012, 2012 International Symposium on Computer, Consumer and Control.

[56]  Antonio Albiol,et al.  Detection of Parked Vehicles Using Spatiotemporal Maps , 2011, IEEE Transactions on Intelligent Transportation Systems.

[57]  Carlo S. Regazzoni,et al.  Real-time video-shot detection for scene surveillance applications , 2000, IEEE Trans. Image Process..

[58]  Jake K. Aggarwal,et al.  Detection of abandoned objects in crowded environments , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[59]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[60]  Kang-Hyun Jo,et al.  Detecting abandoned objects in crowded scenes of surveillance videos using adaptive dual background model , 2015, 2015 8th International Conference on Human System Interaction (HSI).

[61]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[62]  Carlos Orrite-Uruñuela,et al.  Automatic left luggage detection and tracking using multi-camera ukf , 2006 .

[63]  Sridha Sridharan,et al.  Abandoned object detection using multi-layer motion detection , 2008 .

[64]  Liang-Gee Chen,et al.  Localized Detection of Abandoned Luggage , 2010, EURASIP J. Adv. Signal Process..

[65]  Juan José Pantrigo,et al.  Abandoned Object Detection on Controlled Scenes Using Kinect , 2013, IWINAC.

[66]  Jian Zhang,et al.  Detecting New Stable Objects In Surveillance Video , 2005, 2005 IEEE 7th Workshop on Multimedia Signal Processing.

[67]  Sharath Pankanti,et al.  Robust abandoned object detection using region-level analysis , 2011, 2011 18th IEEE International Conference on Image Processing.

[68]  Sharath Pankanti,et al.  Relative Attributes for Large-Scale Abandoned Object Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[69]  Bin Ran,et al.  A Novel Abandoned Object Detection System Based on Three-Dimensional Image Information , 2015, Sensors.

[70]  Rita Cucchiara,et al.  Video Surveillance Online Repository (ViSOR): an integrated framework , 2010, Multimedia Tools and Applications.

[71]  Takeo Kanade,et al.  Layered detection for multiple overlapping objects , 2002, Object recognition supported by user interaction for service robots.

[72]  Chao Zhang,et al.  Abandoned Objects Detection Using Double Illumination Invariant Foreground Masks , 2010, 2010 20th International Conference on Pattern Recognition.

[73]  Liang-Gee Chen,et al.  A Localized Approach to Abandoned Luggage Detection with Foreground-Mask Sampling , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[74]  Massimo Massa,et al.  A New Method for Real Time Abandoned Object Detection and Owner Tracking , 2006, 2006 International Conference on Image Processing.

[75]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  Fatih Murat Porikli,et al.  Changedetection.net: A new change detection benchmark dataset , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[77]  Touradj Ebrahimi,et al.  Video object extraction based on adaptive background and statistical change detection , 2000, IS&T/SPIE Electronic Imaging.

[78]  Carlo S. Regazzoni,et al.  A distributed surveillance system for detection of abandoned objects in unmanned railway environments , 2000, IEEE Trans. Veh. Technol..

[79]  i-LIDS Team,et al.  Imagery Library for Intelligent Detection Systems (i-LIDS); A Standard for Testing Video Based Detection Systems , 2006, Proceedings 40th Annual 2006 International Carnahan Conference on Security Technology.

[80]  David Harwood.  A statistical approach for real time robust background subtraction , 1999 .

[81]  Kuo-Chin Fan,et al.  Suspicious Object Detection and Robbery Event Analysis , 2007, 2007 16th International Conference on Computer Communications and Networks.

[82]  José María Martínez Sanchez,et al.  Robust Unattended and Stolen Object Detection by Fusing Simple Algorithms , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[83]  Rui Wang,et al.  Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[84]  Kevin Smith,et al.  Detecting Abandoned Luggage Items in a Public Space , 2006 .

[85]  A. A. Morozov.  Development of a method for intelligent video monitoring of abnormal behavior of people based on parallel object-oriented logic programming , 2015, Pattern Recognition and Image Analysis.

[86]  Yuri Ivanov,et al.  Robust Abandoned Object Detection Using Dual Foregrounds , 2008, EURASIP J. Adv. Signal Process..

[87]  Raúl Mohedano,et al.  On the Mahalanobis Distance Classification Criterion for Multidimensional Normal Distributions , 2013, IEEE Transactions on Signal Processing.

[88]  Rogério Schmidt Feris,et al.  Robust Detection of Abandoned and Removed Objects in Complex Surveillance Videos , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[89]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[91]  Takashi Toriu,et al.  Unattended object intelligent analyzer for consumer video surveillance , 2011, IEEE Transactions on Consumer Electronics.

[92]  Fatih Murat Porikli,et al.  Detection of temporarily static regions by processing video at different frame rates , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[93]  Wen-Hao Wang,et al.  An Abandoned Objects Management System Based on the Gaussian Mixture Model , 2008, 2008 International Conference on Convergence and Hybrid Information Technology.

[94]  Jagdish Lal Raheja,et al.  Detection of abandoned objects in real time , 2011, 2011 3rd International Conference on Electronics Computer Technology.

[95]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[96]  S. Goldsack,et al.  IN REAL-TIME , 2008 .

[97]  K. N. Ramachandran Nair,et al.  Abandoned or Removed Objects Detection from Surveillence Video using Codebook , 2013 .

[98]  Carlos Cuevas,et al.  Detection of static moving objects using multiple nonparametric background models , 2015, 2015 International Symposium on Consumer Electronics (ISCE).

[99]  Anand Singh Jalal,et al.  A framework for abandoned object detection from video surveillance , 2013, 2013 Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG).

[100]  Hyeran Byun,et al.  Abandoned luggage detection using a finite state automaton in surveillance video , 2010 .

[101]  Vittorio Murino,et al.  Background Subtraction for Automated Multisensor Surveillance: A Comprehensive Review , 2010, EURASIP J. Adv. Signal Process..

[102]  Shireen Elhabian,et al.  Moving Object Detection in Spatial Domain using Background Removal Techniques - State-of-Art , 2008 .

[103]  Michael J. Brooks,et al.  Detecting suspicious background changes in video surveillance of busy scenes , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[104]  Sergio A. Velastin,et al.  PRISMATICA: toward ambient intelligence in public transport environments , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[105]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[106]  Jean-Marc Odobez,et al.  Multi-Layer Background Subtraction Based on Color and Texture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[107]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[108]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[109]  Bong-Nam Kang,et al.  Nonparametric state machine with multiple features for abnormal object classification , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[110]  Xia Zhang,et al.  Generative model for abandoned object detection , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[111]  Gian Luca Foresti,et al.  Automatic detection and indexing of video-event shots for surveillance applications , 2002, IEEE Trans. Multim..

[112]  Rita Cucchiara,et al.  Predictive and Probabilistic Tracking to Detect Stopped Vehicles , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[113]  Sadiye Guler,et al.  Stationary objects in multiple object tracking , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[114]  Léon J. M. Rothkrantz,et al.  Surveillance system using abandoned object detection , 2011, CompSysTech '11.

[115]  José A. Rodríguez-Serrano,et al.  Robust abandoned object detection integrating wide area visual surveillance and social context , 2013, Pattern Recognit. Lett..

[116]  Gustavo Carvalho,et al.  An annotated video database for abandoned-object detection in a cluttered environment , 2014, 2014 International Telecommunications Symposium (ITS).

[117]  José María Martínez Sanchez,et al.  Stationary foreground detection using background subtraction and temporal difference in video surveillance , 2010, 2010 IEEE International Conference on Image Processing.

[118]  Larry S. Davis,et al.  A One-Threshold Algorithm for Detecting Abandoned Packages Under Severe Occlusions Using a Single Camera , 2006 .

[119]  J. Crowley,et al.  CAVIAR Context Aware Vision using Image-based Active Recognition , 2005 .

[120]  Juan C. SanMiguel,et al.  Stationary foreground detection for video-surveillance based on foreground and motion history images , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[121]  Jia-Shung Wang,et al.  A crowd-filter for detection of abandoned objects in crowded area , 2008, 2008 3rd International Conference on Sensing Technology.

[122]  Takashi Toriu,et al.  A Probability-based Model for Detecting Abandoned Objects in Video Surveillance Systems , 2012 .

[123]  Björn Schuller,et al.  Multi-Camera Person Tracking and Left Luggage Detection Applying Homographic Transformation , 2007 .

[124]  Montse Pardàs,et al.  Segmentation and tracking of static and moving objects in video surveillance scenarios , 2008, 2008 15th IEEE International Conference on Image Processing.

[125]  Luigi di Stefano,et al.  Coarse-to-fine strategy for robust and efficient change detectors , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[126]  Michele Magno,et al.  Multimodal Abandoned/Removed Object Detection for Low Power Video Surveillance Systems , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[127]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[128]  Zhang Qin,et al.  An Abandoned Object Detection System Based on Dual Background and Motion Analysis , 2012, 2012 International Conference on Computer Science and Service System.

[129]  Usha A. Joglekar,et al.  An Abandoned object Detection System using Background Segmentation , 2014 .

[130]  Jake K. Aggarwal,et al.  Real-Time Illegal Parking Detection in Outdoor Environments Using 1-D Transformation , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[131]  Osama Masoud,et al.  Real time, online detection of abandoned objects in public areas , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[132]  Xiang Ma,et al.  Learning a background model for change detection , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[133]  Lucia Maddalena,et al.  The SOBS algorithm: What are the limits? , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[134]  Yi-Ping Hung,et al.  Abandoned Object Detection via Temporal Consistency Modeling and Back-Tracing Verification for Visual Surveillance , 2015, IEEE Transactions on Information Forensics and Security.

[135]  David M. Lane,et al.  Human behaviour recognition in data-scarce domains , 2015, Pattern Recognit..

[136]  Michael D. Beynon,et al.  Detecting abandoned packages in a multi-camera video surveillance system , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..