A Directional-Edge-Based Real-Time Object Tracking System Employing Multiple Candidate-Location Generation

We present a directional-edge-based object tracking system based on a field-programmable gate array (FPGA) that can process 640 × 480 resolution video sequences and provide the location of a predefined object in real time. Inspired by biological principle, directional edge information is used to represent the object features. Multiple candidate regeneration, a statistical method, has been developed to realize the tracking function, and online learning is adopted to enhance the tracking performance. Thanks to the hardware-implementation friendliness of the algorithm, an object tracking system has been very efficiently built on an FPGA, in order to realize a real-time tracking capability. At the working frequency of 60 MHz, the main processing circuit can complete the processing of one frame of an image (640 × 480 pixels) in 0.1 ms in high-speed mode and 0.8 ms in high-accuracy mode. The experimental results demonstrate that this system can deal with various complex situations, including scene illumination changes, object deformation, and partial occlusion. Based on the system built on the FPGA, we discuss the issue of very large-scale integrated chip implementation of the algorithm and self initialization of the system, i.e., the autonomous localization of the tracking object in the initial frame. Some potential solutions to the problems of multiple object tracking and full occlusion are also presented.

[1]  Brian C. Lovell,et al.  Robust object tracking using local oriented energy features and its hardware/software implementation , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[2]  Yung-Chang Chen,et al.  High-Performance SIFT Hardware Accelerator for Real-Time Image Feature Extraction , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Shai Avidan,et al.  Ensemble Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jae Wook Jeon,et al.  A Real-Time Object Tracking System Using a Particle Filter , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Hongjun Wang,et al.  Real-Time Object Tracking System on FPGAs , 2011, 2011 Symposium on Application Accelerators in High-Performance Computing.

[6]  Shai Avidan,et al.  Support Vector Tracking , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Chun-Fu Shen,et al.  Real time color based particle filtering for object tracking with dual cache architecture , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[8]  Ming-Hsuan Yang,et al.  Robust Object Tracking with Online Multiple Instance Learning , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[10]  Z W Pylyshyn,et al.  Tracking multiple independent targets: evidence for a parallel tracking mechanism. , 1988, Spatial vision.

[11]  Tadashi Shibata,et al.  A real-time image recognition system using a global directional-edge-feature extraction VLSI processor , 2009, 2009 Proceedings of ESSCIRC.

[12]  C.-C. Jay Kuo,et al.  Efficient VLSI design for SIFT feature description , 2010, 2010 International Symposium on Next Generation Electronics.

[13]  Directional-edge-based object tracking employing on-line learning and regeneration of multiple candidate locations , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[14]  Chen-Chien Hsu,et al.  Hardware/software co-design of particle filter and its application in object tracking , 2011, Proceedings 2011 International Conference on System Science and Engineering.

[15]  Chiou-Ting Hsu,et al.  Online Selection of Tracking Features using AdaBoost , 2007, 2007 16th International Conference on Computer Communications and Networks.

[16]  Imbaby I. Mahmoud,et al.  Proposed hardware architectures of particle filter for object tracking , 2012, EURASIP J. Adv. Signal Process..

[17]  Tadashi Shibata,et al.  An image representation algorithm compatible with neural-associative-processor-based hardware recognition systems , 2003, IEEE Trans. Neural Networks.

[18]  Simon J. Godsill,et al.  On sequential simulation-based methods for Bayesian filtering , 1998 .

[19]  Peihua Li.  An Adaptive Binning Color Model for Mean Shift Tracking , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Tadashi Shibata,et al.  A Pixel-Parallel Self-Similitude Processing for Multiple-Resolution Edge-Filtering Analog Image Sensors , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[21]  Henry Medeiros,et al.  A parallel color-based particle filter for object tracking , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[22]  Johnny Park,et al.  A Parallel Implementation of the Color-Based Particle Filter for Object Tracking , 2008 .

[23]  T. Morimoto,et al.  A fully-parallel vector quantization processor for real-time motion picture compression , 1997, 1997 IEEE International Solids-State Circuits Conference. Digest of Technical Papers.

[24]  Tadashi Shibata,et al.  Multiple-clue face detection algorithm using edge-based feature vectors , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  Irene Y. H. Gu,et al.  Robust Visual Object Tracking Using Multi-Mode Anisotropic Mean Shift and Particle Filters , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Pheng-Ann Heng,et al.  Two-Stage Object Tracking Method Based on Kernel and Active Contour , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Bohyung Han,et al.  Visual Tracking by Continuous Density Propagation in Sequential Bayesian Filtering Framework , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  S. Fazli,et al.  Particle Filter Based Object Tracking with Sift and Color Feature , 2009, 2009 Second International Conference on Machine Vision.

[29]  Andreas E. Savakis,et al.  Online Distance Metric Learning for Object Tracking , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  ZuWhan Kim.  Real time object tracking based on dynamic feature grouping with background subtraction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Ryusuke Miyamoto,et al.  An efficient prediction scheme for pedestrian tracking with cascade particle filter and its implementation on Cell/B.E. , 2009, 2009 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS).

[32]  Yasushi Yagi,et al.  Integrating Color and Shape-Texture Features for Adaptive Real-Time Object Tracking , 2008, IEEE Transactions on Image Processing.

[33]  Songyu Yu,et al.  FPGA-based real-time object tracking for mobile robot , 2010, 2010 International Conference on Audio, Language and Image Processing.

[34]  David Suter,et al.  Adaptive Object Tracking Based on an Effective Appearance Filter , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[36]  Tadashi Shibata,et al.  Soft-Computing Integrated Circuits for Intelligent Information Processing , 1999 .