Parallel Architecture for Hierarchical Optical Flow Estimation Based on FPGA

The proposed work presents a highly parallel architecture for motion estimation. Our system implements the well-known Lucas and Kanade algorithm with the multi-scale extension for the computation of large motion estimations in a dedicated device [field-programmable gate array (FPGA)]. Our system achieves 270 frames per second for a 640 × 480 resolution in the best case of the mono-scale implementation and 32 frames per second for the multi-scale one, fulfilling the requirements for a real-time system. We describe the system architecture, address the evaluation of the accuracy with well-known benchmark sequences (including a comparative study), and show the main hardware resources used.

[1]  Dah-Jye Lee,et al.  A Fast and Accurate Tensor-based Optical Flow Algorithm Implemented in FPGA , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[2]  Marc M. Van Hulle,et al.  Realtime phase-based optical flow on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Adel M. Alimi,et al.  Event Detection from Video Surveillance Data Based on Optical Flow Histogram and High-level Feature Extraction , 2009, 2009 20th International Workshop on Database and Expert Systems Application.

[4]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[5]  Javier Díaz,et al.  Superpipelined high-performance optical-flow computation architecture , 2008, Comput. Vis. Image Underst..

[6]  Paul Merrell,et al.  Structure from motion using optical flow probability distributions , 2005, SPIE Defense + Commercial Sensing.

[7]  Eero P. Simoncelli.  Design of multi-dimensional derivative filters , 1994, Proceedings of 1st International Conference on Image Processing.

[8]  Harry Shum,et al.  Full-frame video stabilization with motion inpainting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Frank Dellaert,et al.  Fast Image-Based Tracking by Selective Pixel Integration , 2011 .

[10]  Anil C. Kokaram,et al.  On missing data treatment for degraded video and film archives: a survey and a new Bayesian approach , 2004, IEEE Transactions on Image Processing.

[11]  Klaus Diepold,et al.  The impact of nonlinear filtering and confidence information on optical flow estimation in a Lucas & Kanade framework , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[12]  Dah-Jye Lee,et al.  Real-Time Optical Flow Calculations on FPGA and GPU Architectures: A Comparison Study , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.

[13]  Hiroaki Niitsuma,et al.  High Speed Computation of the Optical Flow , 2005, ICIAP.

[14]  Marc M. Van Hulle,et al.  Optimal instantaneous rigid motion estimation insensitive to local minima , 2006, Comput. Vis. Image Underst..

[15]  Jonathan W. Brandt,et al.  Improved Accuracy in Gradient-Based Optical Flow Estimation , 1997, International Journal of Computer Vision.

[16]  Eduardo Ros Vidal,et al.  Robust Bioinspired Architecture for Optical-Flow Computation , 2010, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[17]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[18]  Masahiko Yoshimoto,et al.  A VGA 30-fps Realtime Optical-Flow Processor Core for Moving Picture Recognition , 2008, IEICE Trans. Electron..

[19]  Mancia Anguita,et al.  Optimization Strategies for High-Performance Computing of Optical-Flow in General-Purpose Processors , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[21]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[22]  Alan Yuille,et al.  Active Vision , 2014, Computer Vision, A Reference Guide.

[23]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[24]  Timo Kohlberger,et al.  A Multigrid Platform for Real-Time Motion Computation with Discontinuity-Preserving Variational Methods , 2006, International Journal of Computer Vision.

[25]  D HagerGregory,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998 .

[26]  Robin R. Murphy,et al.  A VLSI Architecture and Algorithm for Lucas–Kanade-Based Optical Flow Computation , 2010, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[27]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[28]  Hans-Peter Seidel,et al.  High Accuracy Optical Flow Serves 3-D Pose Tracking: Exploiting Contour and Flow Based Constraints , 2006, ECCV.

[29]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[30]  Antonio Cañas,et al.  Hardware description of multi-layer perceptrons with different abstraction levels , 2006, Microprocess. Microsystems.

[31]  Javier Díaz,et al.  Visual System Based on Artificial Retina for Motion Detection , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[32]  Marco Straforini,et al.  The Use of Optical Flow for the Autonomous Navigation , 1992, Int. J. Neural Syst..

[33]  Joachim Weickert,et al.  Variational optic flow on the Sony PlayStation 3 , 2010, Journal of Real-Time Image Processing.

[34]  Javier Díaz,et al.  Multi-port abstraction layer for FPGA intensive memory exploitation applications , 2010, J. Syst. Archit..

[35]  Florentin Wörgötter,et al.  A cortical architecture on parallel hardware for motion processing in real time. , 2010, Journal of vision.

[36]  K. Nakayama,et al.  Optical Velocity Patterns, Velocity-Sensitive Neurons, and Space Perception: A Hypothesis , 1974, Perception.

[37]  Marina Kolesnik,et al.  Estimation of travel distance from visual motion in virtual environments , 2007, TAP.

[38]  Julien Marzat,et al.  Real-Time Dense and Accurate Parallel Optical Flow using CUDA , 2009 .

[39]  Eduardo Ros,et al.  High-Performance Optical-Flow Architecture Based on a Multi-Scale, Multi-Orientation Phase-Based Model , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Javier Díaz,et al.  FPGA-based real-time optical-flow system , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Timo Kohlberger,et al.  Domain decomposition for variational optical-flow computation , 2005, IEEE Transactions on Image Processing.

[42]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Hans-Hellmut Nagel,et al.  Optical Flow Estimation: Advances and Comparisons , 1994, ECCV.

[44]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[45]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.