Robust Monocular Epipolar Flow Estimation

We consider the problem of computing optical flow in monocular video taken from a moving vehicle. In this setting, the vast majority of image flow is due to the vehicle's ego-motion. We propose to take advantage of this fact and estimate flow along the epipolar lines of the egomotion. Towards this goal, we derive a slanted-plane MRF model which explicitly reasons about the ordering of planes and their physical validity at junctions. Furthermore, we present a bottom-up grouping algorithm which produces over-segmentations that respect flow boundaries. We demonstrate the effectiveness of our approach in the challenging KITTI flow benchmark [11] achieving half the error of the best competing general flow algorithm and one third of the error of the best epipolar flow algorithm.

[1]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[2]  Berthold K. P. Horn,et al.  "Determining optical flow": A Retrospective , 1993, Artif. Intell..

[3]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[4]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[6]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[7]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[8]  Radim Sára,et al.  Stratified Dense Matching for Stereopsis in Complex Scenes , 2003, BMVC.

[9]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[10]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[11]  Joachim Weickert,et al.  Optic Flow Goes Stereo: A Variational Method for Estimating Discontinuity-Preserving Dense Disparity Maps , 2005, DAGM-Symposium.

[12]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[13]  Radim Sára,et al.  Efficient Sampling of Disparity Space for Fast And Accurate Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  H. Hirschmüller Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Joachim Weickert,et al.  A Variational Model for the Joint Recovery of the Fundamental Matrix and the Optical Flow , 2008, DAGM-Symposium.

[16]  Carsten Rother,et al.  FusionFlow: Discrete-continuous optimization for optical flow estimation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Daniel Cremers,et al.  Continuous Energy Minimization Via Repeated Binary Fusion , 2008, ECCV.

[18]  Nassir Navab,et al.  Optical flow estimation with uncertainties through dynamic MRFs , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Cheng Lei,et al.  Optical flow estimation on coarse-to-fine region-trees using discrete optimization , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Daniel Cremers,et al.  Structure- and motion-adaptive regularization for high accuracy optic flow , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Julian Eggert,et al.  A Two-Stage Correlation Method for Stereoscopic Depth Estimation , 2010, 2010 International Conference on Digital Image Computing: Techniques and Applications.

[23]  Andreas Geiger,et al.  Efficient Large-Scale Stereo Matching , 2010, ACCV.

[24]  Tamir Hazan,et al.  Norm-Product Belief Propagation: Primal-Dual Message-Passing for Approximate Inference , 2009, IEEE Transactions on Information Theory.

[25]  S. Süsstrunk,et al.  SLIC Superpixels ? , 2010 .

[26]  Tamir Hazan,et al.  A Primal-Dual Message-Passing Algorithm for Approximated Large Scale Structured Prediction , 2010, NIPS.

[27]  Pushmeet Kohli,et al.  Surface stereo with soft segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Radu Horaud,et al.  Scene flow estimation by growing correspondence seeds , 2011, CVPR 2011.

[29]  David A. McAllester,et al.  Convex max-product algorithms for continuous MRFs with applications to protein folding , 2011, ICML 2011.

[30]  Marc Pollefeys,et al.  Distributed message passing for large scale graphical models , 2011, CVPR 2011.

[31]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[32]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Horst Bischof,et al.  Pushing the limits of stereo using variational stereo estimation , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[34]  Reinhard Klette,et al.  Hierarchical Scan-Line Dynamic Programming for Optical Flow Using Semi-Global Matching , 2012, ACCV Workshops.

[35]  Tamir Hazan,et al.  Continuous Markov Random Fields for Robust Stereo Estimation , 2012, ECCV.

[36]  Henning Lategahn,et al.  Trinocular optical flow estimation for intelligent vehicle applications , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[37]  Reinhard Klette,et al.  Iterative Semi-Global Matching for Robust Driver Assistance Systems , 2012, ACCV.

[38]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Pratim Ghosh,et al.  Robust Simultaneous Registration and Segmentation with Sparse Error Reconstruction , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.