Evaluation of Change Detection Algorithms using Difficulty Maps

The evaluation of a change detection algorithm should show its superiority over state-of-the-art algorithms' performances. Evaluating an algorithm involves executing it to segment a set of videos and comparing the results with the ground truth. Here, we used the difficulty level to classify each pixel of each frame of the videos of a dataset as an algorithm performance measure. A structure called "difficulty map" stores information about the difficulty of classifying each pixel in a frame. Based on these maps, we developed a metric that aims to evaluate the performance of algorithms on the difficulty map. The results showed that there are algorithms with the characteristic of classifying pixels that most state-of-the-art algorithms cannot classify (promising algorithms). Identifying such algorithms is essential since improving their performance means facing challenges already overcome by existing approaches.

[1]  R. A. Arias Velásquez,et al.  Methodology for an automatic license plate recognition system using Convolutional Neural Networks for a Peruvian case study , 2022, IEEE Latin America Transactions.

[2]  S. R. R. Sanches,et al.  Method for selecting representative videos for change detection datasets , 2021, Multimedia Tools and Applications.

[3]  Silvio Ricardo Rodrigues Sanches,et al.  Recommendations for evaluating the performance of background subtraction algorithms for surveillance systems , 2020, Multimedia Tools and Applications.

[4]  Vitor Bruno de Oliveira Barth,et al.  Vehicle Speed Monitoring using Convolutional Neural Networks , 2019, IEEE Latin America Transactions.

[5]  J. Konrad,et al.  BSUV-Net: A Fully-Convolutional Neural Network for Background Subtraction of Unseen Videos , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[6]  S. R. R. Sanches,et al.  Challenging situations for background subtraction algorithms , 2018, Applied Intelligence.

[7]  Long Ang Lim,et al.  Learning multi-scale features for foreground segmentation , 2018, Pattern Analysis and Applications.

[8]  Gerhard Rigoll,et al.  A deep convolutional neural network for video sequence background subtraction , 2018, Pattern Recognit..

[9]  Sahin Isik,et al.  SWCD: a sliding window and self-regulated learning-based background updating method for change detection in videos , 2018, J. Electronic Imaging.

[10]  Fei-Yue Wang,et al.  $M^{4}CD$ : A Robust Change Detection Method for Intelligent Visual Surveillance , 2018, IEEE Access.

[11]  Marc Van Droogenbroeck,et al.  Semantic background subtraction , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[12]  Zhiming Luo,et al.  Interactive deep learning method for segmenting moving objects , 2017, Pattern Recognit. Lett..

[13]  S. Bianco,et al.  How Far Can You Get By Combining Change Detection Algorithms? , 2015, ICIAP.

[14]  Xiqun Lu,et al.  A multiscale spatio-temporal background model for motion detection , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[15]  Bin Wang,et al.  A Fast Self-Tuning Background Subtraction Algorithm , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[16]  Antoine Vacavant,et al.  A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos , 2014, Comput. Vis. Image Underst..

[17]  S. R. R. Sanches,et al.  Bilayer Segmentation of Live Video in Uncontrolled Environments for Background Substitution: An Overview and Main Challenges , 2012, IEEE Latin America Transactions.

[18]  Fatih Murat Porikli,et al.  Changedetection.net: A new change detection benchmark dataset , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[19]  E. O. T. Salles,et al.  A New Change Detection Algorithm for Visual Surveillance System , 2012, IEEE Latin America Transactions.