Parallelization of the optical flow computation in sequences from moving cameras

AbstractThis paper presents a flexible and scalable approach to the parallelization of the computation of optical flow. This approach is based on data parallel distribution. Images are divided into several subimages processed by a software pipeline while respecting dependencies between computation stages. The parallelization has been implemented in three different infrastructures: shared, distributed memory, and hybrid to show its conceptual flexibility and scalability. A significant improvement in performance was obtained in all three cases. These versions have been used to compute the optical flow of video sequences taken in adverse conditions, with a moving camera and natural-light conditions, on board a conventional vehicle traveling on public roads. The parallelization adopted has been developed from the analysis of dependencies presented by the well-known Lucas-Kanade algorithm, using a sequential version developed at the University of Porto as the starting point.

[1]  Hujun Bao,et al.  Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jörg Kramer.  Compact Integrated Motion Sensor With Three-Pixel Interaction , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Alan A. Stocker,et al.  Computation of Smooth Optical Flow in a Feedback Connected Analog Network , 1998, NIPS.

[4]  F. Champagnat,et al.  Fast and accurate PIV computation using highly parallel iterative correlation maximization , 2011 .

[5]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[6]  Han Wang,et al.  A fast algorithm for computing optic flow and its implementation on a Transputer array , 1990, BMVC.

[7]  M.V. Correia,et al.  Optical flow techniques applied to the calibration of visual perception experiments , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[8]  Pietro Perona,et al.  Scene segmentation from 3D motion , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[9]  Dah-Jye Lee,et al.  Accurate Optical Flow Sensor for Obstacle Avoidance , 2008, ISVC.

[10]  T.M. McGinnity,et al.  An Investigation into Optical Flow Computation on FPGA Hardware , 2008, 2008 International Machine Vision and Image Processing Conference.

[11]  C. Laugier,et al.  Real-time moving obstacle detection using optical flow models , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[12]  Liang-Gee Chen,et al.  Robust moving object tracking and trajectory prediction for visual navigation in dynamic environments , 2012, 2012 IEEE International Conference on Consumer Electronics (ICCE).

[13]  Aurélio J. C. Campilho,et al.  A Pipelined Real-Time Optical Flow Algorithm , 2004, ICIAR.

[14]  B. Crespi,et al.  Real-time parallel computation of disparity and optical flow using phase difference , 1996 .

[15]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  John S. Zelek,et al.  Towards Bayesian real-time optical flow , 2004, Image Vis. Comput..

[17]  Antonio García Dopico,et al.  Distributed Computation of Optical Flow , 2004, International Conference on Computational Science.

[18]  Erik Ringaby,et al.  Optical Flow Computation on Compute Unified Device Architecture , 2008 .

[19]  Yoshiki Mizukami,et al.  Optical Flow Computation on Compute Unified Device Architecture , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[20]  Jose Antonio Boluda,et al.  Change-driven data flow image processing architecture for optical flow computation , 2007, Journal of Real-Time Image Processing.

[21]  Dah-Jye Lee,et al.  Real-Time Optical Flow Calculations on FPGA and GPU Architectures: A Comparison Study , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.

[22]  A. Del Bimbo,et al.  Optical flow estimation on Connection-Machine 2 , 1993, 1993 Computer Architectures for Machine Perception.

[23]  Jose Hugo Barron-Zambrano,et al.  Parallel Processor for 3D Recovery from Optical Flow , 2008, ReConFig.

[24]  Edward H. Adelson,et al.  Probability distributions of optical flow , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Sergiu Nedevschi,et al.  Optical flow A combined local-global approach using L1 norm , 2010, Proceedings of the 2010 IEEE 6th International Conference on Intelligent Computer Communication and Processing.

[26]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[27]  Selim Temizer.  Optical flow based local navigation , 2001 .

[28]  Andy C. Downton,et al.  Evaluating optical-flow algorithms on a parallel machine , 2001, Image Vis. Comput..

[29]  Sovira Tan,et al.  Inverse perspective mapping and optic flow: A calibration method and a quantitative analysis , 2006, Image Vis. Comput..

[30]  Hiroaki Niitsuma,et al.  High Speed Computation of the Optical Flow , 2005, ICIAP.

[31]  Rama Chellappa,et al.  Accuracy vs Efficiency Trade-offs in Optical Flow Algorithms , 1996, Comput. Vis. Image Underst..

[32]  V. Nagarajan,et al.  FPGA Based High Performance Optical Flow Computation Using Parallel Architecture , 2012 .

[33]  Stefano Cagnoni,et al.  Cellular automata-based optical flow computation for "just-in-time" applications , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[34]  Seth Teller,et al.  Video matching , 2004, SIGGRAPH 2004.

[35]  Christer Svensson,et al.  Single-Chip High-Speed Computation of Optical Flow , 1990, MVA.

[36]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[37]  Ulrich Rüde,et al.  3D optical flow computation using a parallel variational multigrid scheme with application to cardiac C-arm CT motion , 2007, Image Vis. Comput..

[38]  Shorin Kyo,et al.  Overtaking Vehicle Detection Method and Its Implementation Using IMAPCAR Highly Parallel Image Processor , 2007, IEICE Trans. Inf. Syst..

[39]  Javier Díaz,et al.  Superpipelined high-performance optical-flow computation architecture , 2008, Comput. Vis. Image Underst..

[40]  A. Borst,et al.  Eigenanalysis of a neural network for optic flow processing , 2008 .

[41]  Aurélio J. C. Campilho,et al.  Cellular neural networks for motion estimation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[42]  J. Weickert,et al.  Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods , 2005 .

[43]  Richard G. Lane,et al.  Determining optical flow using a differential method , 1997, Image Vis. Comput..

[44]  Aurélio J. C. Campilho,et al.  Real-time implementation of an optical flow algorithm , 2002, Object recognition supported by user interaction for service robots.

[45]  Marco Platzner,et al.  Specialized Architectures for Optical Flow Computation: A Performance Comparison of ASIC, DSP, and Multi-DSP , 1997 .

[46]  Julien Marzat,et al.  Real-Time Dense and Accurate Parallel Optical Flow using CUDA , 2009 .

[47]  Marc M. Van Hulle,et al.  Optic flow from unstable sequences through local velocity constancy maximization , 2009, Image Vis. Comput..

[48]  Timo Kohlberger,et al.  Domain decomposition for variational optical-flow computation , 2005, IEEE Transactions on Image Processing.

[49]  Minglun Gong.  Real-time joint disparity and disparity flow estimation on programmable graphics hardware , 2009, Comput. Vis. Image Underst..

[50]  Alberto Del Bimbo,et al.  A multilayer massively parallel architecture for optical flow computation , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol. IV. Conference D: Architectures for Vision and Pattern Recognition,.

[51]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[52]  Sukeshni N. Tamgade,et al.  Motion Vector Estimation of Video Image by Pyramidal Implementation of Lucas Kanade Optical Flow , 2009, 2009 Second International Conference on Emerging Trends in Engineering & Technology.

[53]  H. Buxton,et al.  Parallel computations of optic flow in early image processing , 1984 .

[54]  Adrian G. Bors,et al.  Smoothing of optical flow using robustified diffusion kernels , 2010, Image Vis. Comput..

[55]  T. Poggio,et al.  A parallel algorithm for real-time computation of optical flow , 1989, Nature.