A high-performance, low-energy FPGA accelerator for correntropy-based feature tracking (abstract only)

Computer-vision and signal-processing applications often require feature tracking to identify and track the motion of different objects (features) across a sequence of images. Numerous algorithms have been proposed, but common measures of similarity for real-time usage are either based on correlation, mean-squared error, or sum of absolute differences, which are not robust enough for safety-critical applications. To improve robustness, a recent feature-tracking algorithm called C-Flow uses correntropy from Information Theoretic Learning to significantly improve signal-to-noise ratio. In this paper, we present an FPGA accelerator for C-Flow that is typically 3.6-8.5x faster than a GPU and show that the FPGA is the only device capable of real-time usage for large features. Furthermore, we show the FPGA accelerator is more appropriate for embedded usage, with energy consumption that is 2.5-22x less than the GPU.

[1]  Jan-Michael Frahm,et al.  Feature tracking and matching in video using programmable graphics hardware , 2007, Machine Vision and Applications.

[2]  Ted Camus.  Real-Time Quantized Optical Flow , 1997, Real Time Imaging.

[3]  C. Laugier,et al.  Real-time moving obstacle detection using optical flow models , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[4]  Deniz Erdogmus,et al.  Feature extraction using information-theoretic learning , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Karl S. Hemmert,et al.  Closing the gap: CPU and FPGA trends in sustainable floating-point BLAS performance , 2004, 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[6]  Jianbo Shi,et al.  Improving KLT in Embedded Systems by Processing Oversampling Video Sequence in Real-Time , 2011, 2011 International Conference on Reconfigurable Computing and FPGAs.

[7]  José Carlos Príncipe,et al.  A Pitch Detector Based on a Generalized Correlation Function , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[8]  Yong Dou,et al.  Optimized Generation of Memory Structure in Compiling Window Operations onto Reconfigurable Hardware , 2007, ARC.

[9]  Maya Gokhale,et al.  Matched Filter Computation on FPGA, Cell and GPU , 2007 .

[10]  Greg Brown,et al.  A performance and energy comparison of FPGAs, GPUs, and multicores for sliding-window applications , 2012, FPGA '12.

[11]  Takeo Kanade,et al.  Realtime affine-photometric KLT feature tracker on GPU in CUDA framework , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[12]  Miriam Leeser,et al.  Adaptable Two-Dimension Sliding Windows on NVIDIA GPUs with Runtime Compilation , 2011, 2011 Symposium on Application Accelerators in High-Performance Computing.

[13]  Ahmed Bouridane,et al.  Design and implementation of a high level programming environment for FPGA-based image processing , 2000 .

[14]  Eduardo Ros,et al.  A Comparison of FPGA and GPU for Real-Time Phase-Based Optical Flow, Stereo, and Local Image Features , 2012, IEEE Transactions on Computers.

[15]  Wayne Luk,et al.  Have GPUs made FPGAs redundant in the field of video processing? , 2005, Proceedings. 2005 IEEE International Conference on Field-Programmable Technology, 2005..

[16]  Dah-Jye Lee,et al.  FPGA-based Real-time Optical Flow Algorithm Design and Implementation , 2007, J. Multim..

[17]  Tsutomu Maruyama,et al.  Performance comparison of FPGA, GPU and CPU in image processing , 2009, 2009 International Conference on Field Programmable Logic and Applications.

[18]  Kevin Skadron,et al.  Accelerating Compute-Intensive Applications with GPUs and FPGAs , 2008, 2008 Symposium on Application Specific Processors.

[19]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[20]  Javier Díaz,et al.  Parallel Architecture for Hierarchical Optical Flow Estimation Based on FPGA , 2012, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[21]  A. Matos,et al.  An Autonomous Boat Based Synthetic Aperture Sonar , 2007, OCEANS 2007.

[22]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[23]  Miriam Leeser,et al.  Automatic Sliding Window Operation Optimization for FPGA-Based , 2006, 2006 14th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[24]  Andrzej Cichocki,et al.  A new nonlinear similarity measure for multichannel signals , 2008, Neural Networks.

[25]  Jan-Michael Frahm,et al.  Fast gain-adaptive KLT tracking on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[26]  Abbes Amira,et al.  FPGA implementations of fast Fourier transforms for real-time signal and image processing , 2005 .

[27]  Dah-Jye Lee,et al.  Real-Time Optical Flow Calculations on FPGA and GPU Architectures: A Comparison Study , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.

[28]  Peter Lee,et al.  An FPGA implementation of a flexible, parallel image processing architecture suitable for embedded vision systems , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[29]  John S. Zelek,et al.  Towards Bayesian real-time optical flow , 2004, Image Vis. Comput..

[30]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[32]  Weifeng Liu,et al.  Correntropy: A Localized Similarity Measure , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[33]  Bao-Gang Hu,et al.  Robust feature extraction via information theoretic learning , 2009, ICML '09.

[34]  Pavlos Protopapas,et al.  An Information Theoretic Algorithm for Finding Periodicities in Stellar Light Curves , 2012, IEEE Transactions on Signal Processing.

[35]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[36]  Nicholas J. Redding,et al.  GPU-Accelerated KLT Tracking with Monte-Carlo-Based Feature Reselection , 2008, 2008 Digital Image Computing: Techniques and Applications.