Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device

The processing of a high-definition video stream in real-time is a challenging task for embedded systems. However, modern FPGA devices have both a high operating frequency and sufficient logic resources to be successfully used in these tasks. In this article, an advanced system that is able to generate and maintain a complex background model for a scene as well as segment the foreground for an HD colour video stream (1,920 × 1,080 @ 60 fps) in real-time is presented. The possible application ranges from video surveillance to machine vision systems. That is, in all cases, when information is needed about which objects are new or moving in the scene. Excellent results are obtained by using the CIE Lab colour space, advanced background representation as well as integrating information about lightness, colour and texture in the segmentation step. Finally, the complete system is implemented in a single high-end FPGA device.

[1]  Tomasz Kryjak,et al.  Real-time moving object detection for video surveillance system in FPGA , 2011, Proceedings of the 2011 Conference on Design & Architectures for Signal & Image Processing (DASIP).

[2]  Surveillance.  Proceedings : 2nd joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS), October 15-16, 2005, Beijing, China , 2005 .

[3]  Raimundo Carlos Silvério Freire,et al.  FPGA architecture for static background subtraction in real time , 2006, SBCCI '06.

[4]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[5]  Shih-Chia Huang,et al.  An Advanced Motion Detection Algorithm With Video Quality Analysis for Video Surveillance Systems , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Eduardo Ros,et al.  FPGA Implementation for Real-Time Background Subtraction Based on Horprasert Model , 2012, Sensors.

[7]  Mohan M. Trivedi,et al.  Detecting Moving Shadows: Algorithms and Evaluation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Csaba Benedek,et al.  Study on color space selection for detecting cast shadows in video surveillance , 2007, Int. J. Imaging Syst. Technol..

[9]  Mubarak Shah,et al.  A hierarchical approach to robust background subtraction using color and gradient information , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[10]  Marek Gorgon,et al.  FPGA-based Road Traffic Videodetector , 2007, 10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007).

[11]  Ettore Napoli,et al.  FPGA-based architecture for real time segmentation and denoising of HD video , 2013, Journal of Real-Time Image Processing.

[12]  Eduardo Ros Vidal,et al.  Codebook hardware implementation on FPGA for background subtraction , 2012, Journal of Real-Time Image Processing.

[13]  Jonathan H. Connell,et al.  A Statistical Approach for Real-time Robust Background Subtrac tion and Shadow Detection , 2014 .

[14]  Liyuan Li,et al.  Integrating intensity and texture differences for robust change detection , 2002, IEEE Trans. Image Process..

[15]  François Brémond,et al.  Shadow Removal in Indoor Scenes , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[16]  Qi Tian,et al.  Foreground object detection from videos containing complex background , 2003, MULTIMEDIA '03.

[17]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Tamás Szirányi,et al.  Study on color space selection for detecting cast shadows in video surveillance: Articles , 2007 .

[19]  Shireen Elhabian,et al.  Moving Object Detection in Spatial Domain using Background Removal Techniques - State-of-Art , 2008 .

[20]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[21]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[22]  Andrew Hunter,et al.  A single-chip FPGA implementation of real-time adaptive background model , 2005, Proceedings. 2005 IEEE International Conference on Field-Programmable Technology, 2005..

[23]  M.P.T. Juvonen,et al.  Hardware Architectures for Adaptive Background Modelling , 2007, 2007 3rd Southern Conference on Programmable Logic.

[24]  Brian C. Lovell,et al.  Improved Shadow Removal for Robust Person Tracking in Surveillance Scenarios , 2010, 2010 20th International Conference on Pattern Recognition.

[25]  Wei Zhang,et al.  Moving Cast Shadows Detection Using Ratio Edge , 2007, IEEE Transactions on Multimedia.

[26]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Nigel J. B. McFarlane,et al.  Segmentation and tracking of piglets in images , 1995, Machine Vision and Applications.

[28]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[29]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[30]  Thierry Bouwmans,et al.  Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey , 2008 .

[31]  M. Gorgon.  Parallel performance of the fine-grain pipeline FPGA image processing system , 2012 .

[32]  Sridha Sridharan,et al.  Real-time adaptive background segmentation , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[33]  Soraia Raupp Musse,et al.  Background Subtraction and Shadow Detection in Grayscale Video Sequences , 2005, XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI'05).

[34]  E.M. Saad,et al.  FPGA-based object-extraction based on multimodal Σ-Δ background estimation , 2009, 2009 2nd International Conference on Computer, Control and Communication.

[35]  Shengcai Liao,et al.  Moving Cast Shadow Removal Based on Local Descriptors , 2010, 2010 20th International Conference on Pattern Recognition.

[36]  Larry S. Davis,et al.  A Robust Background Subtraction and Shadow Detection , 1999 .

[37]  Viktor Öwall,et al.  Hardware accelerator design for video segmentation with multi-modal background modelling , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[38]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..