Memory-based spatio-temporal real-time object segmentation for video surveillance

In real-time content-oriented video applications, fast unsupervised object segmentation is required. This paper proposes a real-time unsupervised object segmentation that is stable throughout large video shots. It trades precise segmentation at object boundaries for speed of execution and reliability in varying image conditions. This interpretation is most appropriate to applications such as surveillance and video retrieval where speed and temporal reliability are of more concern than accurate object boundaries. Both objective and subjective evaluations, and comparisons to other methods show the robustness of the proposed methods while being of reduced complexity. The proposed algorithm needs on average 0.15 seconds per image. The proposed segmentation consists of four steps: motion detection, morphological edge detection, contour analysis, and object labeling. The contributions in this paper are: a segmentation process of simple but effective tasks avoiding complex operations, a reliable memory-based noise-adaptive motion detection, and a memory-based contour tracing and analysis method. The proposed contour tracing aims 1) at finding contours with complex structure such as those containing dead or inner branches and 2) at spatial and temporal adaptive selection of contours. The motion detection is spatio-temporal adaptive as it uses estimated intra-image noise variance and detected inter-image motion.