Image Processing Architectures for Binary Morphology and Labeling

Conventional surveillance systems are omnipresent and most are still based on analog techniques. Migrating to the digital domain grants access to the world of digital image processing enabling automation of such systems, which means extracting information from the image stream without human interaction. The resolution, frame rates, and functionality in these systems are continuously increasing alongside the number of video streams to be processed. The sum of all these parameters imposes high data rates and memory bandwidths which are impossible to handle in pure software solutions. Therefore, accelerating key operations and complex repetitive calculations in dedicated hardware architectures is crucial to sustain real-time performance in future advanced high resolution and frame rate systems. To achieve this goal, this thesis presents four architectures of hardware accelerators to be used in real-time embedded image processing systems, implemented as an FPGA or ASIC. Two morphological architectures performing binary erosion or dilation, with low complexity and low memory requirement, have been developed. One supports static, and the other locally adaptive flat rectangular structuring elements of arbitrary size. Furthermore, a high-throughput architecture calculating the distance transform has also been developed. This architecture supports either the city-block or chessboard distance metric and is based on adding the result of parallel erosions. The fourth architecture performs connected component labeling based on contour tracing and supports feature extraction. A modified version of the morphological architecture supporting static structuring elements, as well as the labeling architecture, has been successfully integrated into a prototype of an automated digital surveillance system for which implementation aspects are presented. The system has been implemented and is running on an FPGA based development board using a CMOS sensor for image acquisition. The prototype currently has segmentation, filtering, and labeling accelerated in hardware, and additional image processing performed in software running on an embedded processor. (Less)

[1]  Kenji Suzuki,et al.  Fast connected-component labeling based on sequential local operations in the course of forward raster scan followed by backward raster scan , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[2]  Edward A. Lee,et al.  DSP Processor Fundamentals , 1997 .

[3]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Dan Schonfeld,et al.  A fast thresholded linear convolution representation of morphological operations , 1994, IEEE Trans. Image Process..

[5]  Marc Van Droogenbroeck,et al.  Fast computation of morphological operations with arbitrary structuring elements , 1996, Pattern Recognit. Lett..

[6]  Kristof Denolf,et al.  A scalable, multi-stream MPEG-4 video decoder for conferencing and surveillance applications , 2005, IEEE International Conference on Image Processing 2005.

[7]  Viktor Öwall,et al.  An Embedded Real-Time Surveillance System: Implementation and Evaluation , 2008, J. Signal Process. Syst..

[8]  G. Matheron.  Random Sets and Integral Geometry , 1976 .

[9]  Ákos Zarándy,et al.  Implementation of binary and gray-scale mathematical morphology on the CNN universal machine , 1996 .

[10]  Shahram Shirani,et al.  ASIC and FPGA implementations of H.264 DCT and quantization blocks , 2005, IEEE International Conference on Image Processing 2005.

[11]  H.M. Wechsler,et al.  Digital image processing, 2nd ed. , 1981, Proceedings of the IEEE.

[12]  Viktor Öwall,et al.  Hardware aspects of a real-time surveillance system , 2006 .

[13]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[14]  Yan Zhang,et al.  Clock-Gating in FPGAs: A Novel and Comparative Evaluation , 2006, 9th EUROMICRO Conference on Digital System Design (DSD'06).

[15]  Roland T. Chin,et al.  Decomposition of Arbitrarily Shaped Morphological Structuring Elements , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  H. Hadwiger,et al.  Minkowskische Addition und Subtraktion beliebiger Punktmengen und die Theoreme von Erhard Schmidt , 1950 .

[17]  Arun N. Netravali,et al.  Digital Pictures: Representation, Compression and Standards , 1995 .

[18]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[19]  Viktor Öwall,et al.  Implementation of a Labeling Algorithm based on Contour Tracing with Feature Extraction , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[20]  Viktor Öwall,et al.  A Custom Image Convolution DSP with a Sustained Calculation Capacity of >1 GMAC/s and Low I/O Bandwidth , 1999, J. VLSI Signal Process..

[21]  Edward R. Dougherty,et al.  Hands-on Morphological Image Processing , 2003 .

[22]  Ioannis Andreadis,et al.  Design and implementation of a fuzzy hardware structure for morphological color image processing , 2003, IEEE Trans. Circuits Syst. Video Technol..

[23]  Thomas Olsson,et al.  Digital implementation of a wavelet-based event detector for cardiac pacemakers , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[24]  Chi-Min Liu,et al.  A new algorithm and its VLSI architecture design for connected component labeling , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[25]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[26]  Luigi di Stefano,et al.  A simple and efficient connected components labeling algorithm , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[27]  Behrooz Parhami,et al.  Computer arithmetic - algorithms and hardware designs , 1999 .

[28]  David W. Paglieroni,et al.  Distance transforms: Properties and machine vision applications , 1992, CVGIP Graph. Model. Image Process..

[29]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  Chun-Jen Chen,et al.  A component-labeling algorithm using contour tracing technique , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[31]  Alberto Broggi,et al.  Decomposition of Arbitrarily Shaped Binary Morphological Structuring Elements Using Genetic Algorithms , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Pierre Soille.  From Binary to Grey Scale Convex Hulls , 2000, Fundam. Informaticae.

[33]  Liang-Gee Chen,et al.  Partial-result-reuse architecture and its design technique for morphological operations with flat structuring elements , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Derek R. Magee,et al.  Tracking multiple vehicles using foreground, background and motion models , 2004, Image Vis. Comput..

[35]  Kenji Suzuki,et al.  Linear-time connected-component labeling based on sequential local operations , 2003, Comput. Vis. Image Underst..

[36]  Hanan Samet,et al.  A general approach to connected-component labeling for arbitrary image representations , 1992, JACM.

[37]  Viktor Öwall,et al.  Hardware accelerator design for video segmentation with multi-modal background modelling , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[38]  N. Sudha,et al.  A pipelined array architecture for Euclidean distance transformation and its FPGA implementation , 2005, Microprocess. Microsystems.

[39]  Viktor Öwall,et al.  Teaching digital HW-design by implementing a complete MP3 decoder , 2003, Proceedings 2003 IEEE International Conference on Microelectronic Systems Education. MSE'03.

[40]  Viktor Öwall,et al.  Background Segmentation Beyond RGB , 2006, ACCV.

[41]  Wayne Luk,et al.  Novel FPGA-based implementation of median and weighted median filters for image processing , 2005, International Conference on Field Programmable Logic and Applications, 2005..

[42]  Steven Trimberger,et al.  A 90-nm Low-Power FPGA for Battery-Powered Applications , 2006, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[43]  Etienne Decencière,et al.  Image filtering using morphological amoebas , 2007, Image Vis. Comput..

[44]  Ching Y. Suen,et al.  Pattern Recognition. The Journal of the Pattern Recognition Society , 1968 .

[45]  Paul Wintz,et al.  Digital image processing (2nd ed.) , 1987 .

[46]  Gunilla Borgefors,et al.  Distance transformations in digital images , 1986, Comput. Vis. Graph. Image Process..

[47]  Azriel Rosenfeld,et al.  Sequential Operations in Digital Picture Processing , 1966, JACM.

[48]  Yazid M. Sharaiha,et al.  Binary digital image processing - a discrete approach , 1999 .

[49]  J. Velten,et al.  Implementation of a high-performance hardware architecture for binary morphological image processing operations , 2004, The 2004 47th Midwest Symposium on Circuits and Systems, 2004. MWSCAS '04..

[50]  W. James MacLean,et al.  An Evaluation of the Suitability of FPGAs for Embedded Vision Systems , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[51]  A. D. Smith,et al.  Color for Philosophers , 1991 .

[52]  Chi-Kwong Li,et al.  A fast algorithm for morphological operations with flat structuring element , 1998 .

[53]  Chun-Jen Chen,et al.  A linear-time component-labeling algorithm using contour tracing technique , 2004, Comput. Vis. Image Underst..

[54]  R. M. Haralick.  Mathematical Morphology And Computer Vision , 1988, Twenty-Second Asilomar Conference on Signals, Systems and Computers.

[55]  Henk J. A. M. Heijmans,et al.  Fundamenta Morphologicae Mathematicae , 2000, Fundam. Informaticae.

[56]  Matthias Kamuf,et al.  A manual on ASIC front to back end design flow , 2005, 2005 IEEE International Conference on Microelectronic Systems Education (MSE'05).

[57]  M. Russo,et al.  A novel class of sorting networks , 1996 .

[58]  Viktor Öwall,et al.  A low complexity architecture for binary image erosion and dilation using structuring element decomposition , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[59]  Theodora A. Varvarigou,et al.  Fast Implementation of Binary Morphological Operations on Hardware-Efficient Systolic Architectures , 2000, J. VLSI Signal Process..

[60]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Sándor Fejes,et al.  A data-driven algorithm and systolic architecture for image morphology , 1994, Proceedings of 1st International Conference on Image Processing.

[62]  Matthias Kamuf,et al.  Teaching digital ASIC design to students with heterogeneous previous knowledge , 2005, 2005 IEEE International Conference on Microelectronic Systems Education (MSE'05).

[63]  Rafael C. González,et al.  Digital image processing using MATLAB , 2006 .

[64]  Anantha P. Chandrakasan,et al.  Minimizing power consumption in digital CMOS circuits , 1995, Proc. IEEE.

[65]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Azriel Rosenfeld,et al.  Connectivity in Digital Pictures , 1970, JACM.

[67]  Ramakant Nevatia,et al.  Tracking multiple humans in complex situations , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Jukka Saarinen,et al.  Hardware architecture for real-time distance transform , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[69]  H. Minkowski.  Volumen und Oberfläche , 1903 .

[70]  Xiaoli Yang,et al.  Watershed segmentation for binary images with different distance transforms , 2004, Proceedings. Second International Conference on Creating, Connecting and Collaborating through Computing.

[71]  L. Råde,et al.  Mathematics handbook for science and engineering , 1995 .

[72]  Viktor Öwall,et al.  Real-Time Video Segmentation with VGA Resolution and Memory Bandwidth Reduction , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[73]  M. Bayoumi,et al.  VLSI architecture for an object change detector for visual sensors , 2005, IEEE Workshop on Signal Processing Systems Design and Implementation, 2005..

[74]  Marcel van Herk.  A fast algorithm for local minimum and maximum filters on rectangular and octagonal kernels , 1992, Pattern Recognit. Lett..

[75]  Fredrik Kristensen.  Design and Implementation of Embedded Video Surveillance Hardware , 2007 .

[76]  Harada Koichi,et al.  A connected component labeling algorithm for grayscale images and application of the algorithm on mammograms , 2007, SAC '07.

[77]  Hugo Hedberg,et al.  A complete MP3 decoder on a chip , 2005, 2005 IEEE International Conference on Microelectronic Systems Education (MSE'05).

[78]  Pierre Soille.  Morphological Operators , 2000, Computer Vision and Applications.

[79]  Andrew G. Dempster,et al.  Using granulometries in processing images of malarial blood , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[80]  Henk J. A. M. Heijmans,et al.  Grey-Scale Morphology Based on Fuzzy Logic , 2002, Journal of Mathematical Imaging and Vision.

[81]  Olivier Cuisenaire.  Locally adaptable mathematical morphology using distance transformations , 2006, Pattern Recognit..

[82]  Anton Kummert,et al.  FPGA-based implementation of variable sized structuring elements for 2D binary morphological operations , 2002, Proceedings First IEEE International Workshop on Electronic Design, Test and Applications '2002.