FPGA-Based Implementation of Real Time Optical Flow Algorithm and Its Applications for Digital Image Stabilization

An efficient simplification procedure of the optical flow (OF) algorithm as well as its hardware implementation using the field programmable gate array (FPGA) technology is presented. The modified algorithm is based on block matching of subsets of successive frames, and exploits one-dimensional representation of subsets as well as the adaptive adjustments of their sizes. Also, an l1-norm-based correlation function requiring no multiplication/division operations is used. As a result, it was possible to reduce the computational complexity of the algorithm without compromising the processing accuracy. Both the accuracy and the limitations resulting from the introduced simplifications have been verified based on several examples of both synthetic and real movie samples. The presented algorithm has been implemented using VirtexII-1000 FPGA to realize a digital stabilization system for the CMOS camera images. Experimental results fully confirm the efficiency of the presented algorithm when working with limited computational resources. This demonstrates the possibility of using our algorithm in the autonomous navigation and other battery-powered systems.

[1]  Timo Kohlberger,et al.  Real-Time Optic Flow Computation with Variational Methods , 2003, CAIP.

[2]  Brendan McCane,et al.  On Benchmarking Optical Flow , 2001, Comput. Vis. Image Underst..

[3]  Klaus Diepold,et al.  Performance of Optical Flow Techniques on Graphics Hardware , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[4]  Nicola Ancona,et al.  Optical flow from 1-D correlation: Application to a simple time-to-crash detector , 1993, 1993 (4th) International Conference on Computer Vision.

[5]  Christopher E. Neely,et al.  Mixed-mode VLSI optic flow sensors for in-flight control of a micro air vehicle , 2000, SPIE Optics + Photonics.

[6]  Javier Díaz,et al.  FPGA-based real-time optical-flow system , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  K. Althoefer,et al.  Optical Flow Algorithm for Velocity Estimation of Ground Vehicles: A Feasibility Study , 2008 .

[8]  Dah-Jye Lee,et al.  A Fast and Accurate Tensor-based Optical Flow Algorithm Implemented in FPGA , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[9]  Klaus Janschek,et al.  Optical Flow Based Navigation for Mobile Robots Using an Embedded Optical Correlator , 2004 .

[10]  S. Erturk,et al.  Digital image stabilization with sub-image phase correlation based global motion estimation , 2003, IEEE Trans. Consumer Electron..

[11]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  Sung-Hee Lee,et al.  Digital Image Stabilizing Algorithms Based On Bit-plane Matching , 1998, International 1998 Conference on Consumer Electronics.

[13]  Dah-Jye Lee,et al.  FPGA-Based Embedded Motion Estimation Sensor , 2008, Int. J. Reconfigurable Comput..

[14]  Xinggang Lin,et al.  Digital image stabilization based on circular block matching , 2006, IEEE Transactions on Consumer Electronics.

[15]  Jangheon Kim,et al.  Hybrid recursive energy-based method for robust optical flow on large motion fields , 2005, IEEE International Conference on Image Processing 2005.

[16]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.