Fast Piecewise-Affine Motion Estimation Without Segmentation

Current algorithmic approaches for piecewise affine motion estimation are based on alternating motion segmentation and estimation. We propose a new method to estimate piecewise affine motion fields directly without intermediate segmentation. To this end, we reformulate the problem by imposing piecewise constancy of the parameter field, and derive a specific proximal splitting optimization scheme. A key component of our framework is an efficient 1D piecewise-affine estimator for vector-valued signals. The first advantage of our approach over segmentation-based methods is its absence of initialization. The second advantage is its lower computational cost, which is independent of the complexity of the motion field. In addition to these features, we demonstrate competitive accuracy with other piecewise-parametric methods on standard evaluation benchmarks. Our new regularization scheme also outperforms the more standard use of total variation and total generalized variation.

[1]  Michael J. Black,et al.  Optical Flow with Semantic Segmentation and Localized Layers , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Wotao Yin,et al.  Global Convergence of ADMM in Nonconvex Nonsmooth Optimization , 2015, Journal of Scientific Computing.

[3]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Daniel Cremers,et al.  Anisotropic Huber-L1 Optical Flow , 2009, BMVC.

[5]  Daniel Cremers,et al.  Near Real-Time Motion Segmentation Using Graph Cuts , 2006, DAGM-Symposium.

[6]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[7]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[8]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[9]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[10]  Andreas Weinmann,et al.  Fast Partitioning of Vector-Valued Images , 2014, SIAM J. Imaging Sci..

[11]  G. Winkler,et al.  Complexity Penalized M-Estimation , 2008 .

[12]  Zheng Xu,et al.  An Empirical Study of ADMM for Nonconvex Problems , 2016, ArXiv.

[13]  Martin Storath,et al.  An algorithmic framework for Mumford–Shah regularization of inverse problems in imaging , 2015 .

[14]  Horst Bischof,et al.  Joint motion estimation and segmentation of complex scenes with label costs and occlusion modeling , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Hui Cheng,et al.  Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection , 2006, ECCV.

[18]  Cordelia Schmid,et al.  DeepFlow: Large Displacement Optical Flow with Deep Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Michael Unser,et al.  Joint image reconstruction and segmentation using the Potts model , 2014, 1405.5850.

[20]  Michael Elad,et al.  Sparsity Based Methods for Overparametrized Variational Problems , 2014, SIAM J. Imaging Sci..

[21]  Cordelia Schmid,et al.  EpicFlow: Edge-preserving interpolation of correspondences for optical flow , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Patrick Pérez,et al.  Hierarchical Estimation and Segmentation of Dense Motion Fields , 2002, International Journal of Computer Vision.

[23]  Martin Storath,et al.  Iterative Potts and Blake–Zisserman minimization for the recovery of functions with discontinuities from indirect measurements , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[24]  Alfred M. Bruckstein,et al.  Over-Parameterized Variational Optical Flow , 2007, International Journal of Computer Vision.

[25]  Jean-Marc Odobez,et al.  Direct incremental model-based image motion segmentation for video analysis , 1998, Signal Process..

[26]  Michael J. Black,et al.  Layered image motion with explicit occlusions, temporal consistency, and depth ordering , 2010, NIPS.

[27]  Mila Nikolova,et al.  Disparity and Optical Flow Partitioning Using Extended Potts Priors , 2014, 1405.1594.

[28]  Stefan Roth,et al.  Joint Optical Flow and Temporally Consistent Semantic Segmentation , 2016, ECCV Workshops.

[29]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[30]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[31]  Carlos Vázquez,et al.  Joint multiregion segmentation and parametric estimation of image motion by basis function representation and level set evolution , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[33]  Thomas Pock,et al.  Non-local Total Generalized Variation for Optical Flow Estimation , 2014, ECCV.

[34]  Jiaolong Yang,et al.  Dense, accurate optical flow estimation with piecewise parametric model , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Daniel Cremers,et al.  An Unbiased Second-Order Prior for High-Accuracy Motion Estimation , 2008, DAGM-Symposium.

[36]  Michael J. Black,et al.  A Naturalistic Open Source Movie for Optical Flow Evaluation , 2012, ECCV.

[37]  Laurent Condat,et al.  A Direct Algorithm for 1-D Total Variation Denoising , 2013, IEEE Signal Processing Letters.

[38]  A. Chambolle FINITE-DIFFERENCES DISCRETIZATIONS OF THE MUMFORD-SHAH FUNCTIONAL , 1999 .

[39]  Margrit Gelautz,et al.  Segmentation-Based Motion with Occlusions Using Graph-Cut Optimization , 2006, DAGM-Symposium.

[40]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[42]  Konrad Schindler,et al.  An Evaluation of Data Costs for Optical Flow , 2013, GCPR.

[43]  Andrew W. Fitzgibbon,et al.  Highly Overparameterized Optical Flow Using PatchMatch Belief Propagation , 2014, ECCV.

[44]  Michael J. Black,et al.  A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them , 2013, International Journal of Computer Vision.

[45]  Rachid Deriche,et al.  Geodesic active regions and level set methods for motion estimation and tracking , 2005, Comput. Vis. Image Underst..

[46]  Konrad Schindler,et al.  3D Scene Flow Estimation with a Piecewise Rigid Scene Model , 2015, International Journal of Computer Vision.

[47]  Andreas Weinmann,et al.  Jump-Sparse and Sparse Recovery Using Potts Functionals , 2013, IEEE Transactions on Signal Processing.

[48]  Michael J. Black,et al.  A Fully-Connected Layered Model of Foreground and Background Flow , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Patrick Bouthemy,et al.  Aggregation of local parametric candidates with exemplar-based occlusion handling for optical flow , 2014, Comput. Vis. Image Underst..

[50]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[51]  Patrick Bouthemy,et al.  Motion segmentation and qualitative dynamic scene analysis from an image sequence , 1993, International Journal of Computer Vision.

[52]  G. Winkler,et al.  Smoothers for Discontinuous Signals , 2002 .

[53]  Janusz Konrad,et al.  Occlusion-Aware Optical Flow Estimation , 2008, IEEE Transactions on Image Processing.

[54]  Brendt Wohlberg,et al.  A nonconvex ADMM algorithm for group sparsity with sparse groups , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[55]  Michael J. Black,et al.  Layered segmentation and optical flow estimation over time , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.