Self-organizing multi-cue fusion for FPGA-based embedded imaging

Self-organization is a natural concept that helps complex systems to adapt themselves autonomically to their environment. In this paper, we present a self-organizing framework for multi-cue fusion in embedded imaging. This means that several simple image filters are used in combination to lead to a more robust system behavior. Human motion tracking serves as a show case. The system adapts to changes in the environment while tracking a person. Besides this, system customization can be simplified. The designer just has to select a desired set of image filters for a given task. The system then finds the appropriate parameters, e.g., the weighting of different cues. With the option of partial re-configuration, FPGAs support this type of customization. An FPGA-based prototype implementation demonstrates the feasibility of this approach. Tracking and adaptation work in real-time with 25 FPS and a resolution of 640×480.

[1]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[2]  Bernt Schiele,et al.  Towards Robust Multi-cue Integration for Visual Tracking , 2001, ICVS.

[3]  Wayne H. Wolf,et al.  Smart Cameras as Embedded Systems , 2002, Computer.

[4]  Jochen Triesch,et al.  Democratic Integration: Self-Organized Integration of Adaptive Cues , 2001, Neural Computation.

[5]  Wolfgang Straßer,et al.  Adaptive Probabilistic Tracking Embedded in Smart Cameras for Distributed Surveillance in a 3D Model , 2007, EURASIP J. Embed. Syst..

[6]  Andrea Prati,et al.  Image convolution on FPGAs: the implementation of a multi-FPGA FIFO structure , 1998, Proceedings. 24th EUROMICRO Conference (Cat. No.98EX204).

[7]  Donald G. Bailey,et al.  FPGA based Remote Object Tracking for Real-time Control , 2005 .

[8]  Jason Schlessman,et al.  Hardware/Software Co-Design of an FPGA-based Embedded Tracking System , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[9]  Anton van den Hengel,et al.  Probabilistic Multiple Cue Integration for Particle Filter Based Tracking , 2003, DICTA.

[10]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[11]  Jürgen Teich,et al.  The Erlangen Slot Machine: A Dynamically Reconfigurable FPGA-based Computer , 2007, J. VLSI Signal Process..

[12]  Wolfgang Straßer,et al.  Adaptive Probabilistic Tracking Embedded in a Smart Camera , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[13]  Giulio Sandini,et al.  A Survey of Artificial Cognitive Systems: Implications for the Autonomous Development of Mental Capabilities in Computational Agents , 2007, IEEE Transactions on Evolutionary Computation.

[14]  C. Malsburg,et al.  Self-organized integration of adaptive visual cues for face tracking , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).