Disparity and Optical Flow Partitioning Using Extended Potts Priors

This paper addresses the problems of disparity and optical flow partitioning based on the brightness invariance assumption. We investigate new variational approaches to these problems with Potts priors and possibly box constraints. For the optical flow partitioning, our model includes vector-valued data and an adapted Potts regularizer. Using the notion of asymptotically level stable (als) functions, we prove the existence of global minimizers of our functionals. We propose a modified alternating direction method of multipliers. This iterative algorithm requires the computation of global minimizers of classical univariate Potts problems which can be done efficiently by dynamic programming. We prove that the algorithm converges both for the constrained and unconstrained problems. Numerical examples demonstrate the very good performance of our partitioning method.

[1]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[2]  Hugues Talbot,et al.  A Majorize-Minimize Subspace Approach for ℓ2-ℓ0 Image Regularization , 2011, SIAM J. Imaging Sci..

[3]  Nebojsa Jojic,et al.  Consistent segmentation for optical flow estimation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[5]  Bin Dong,et al.  ℓ0 Minimization for wavelet frame based image restoration , 2011, Math. Comput..

[6]  Rachid Deriche,et al.  Computing Optical Flow via Variational Techniques , 1999, SIAM J. Appl. Math..

[7]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Horst Bischof,et al.  Joint motion estimation and segmentation of complex scenes with label costs and occlusion modeling , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Étienne Mémin,et al.  Wavelet-Based Fluid Motion Estimation , 2011, SSVM.

[11]  Jean-Christophe Pesquet,et al.  A Convex Optimization Approach for Depth Estimation Under Illumination Variation , 2009, IEEE Transactions on Image Processing.

[12]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[13]  M. Teboulle,et al.  Asymptotic cones and functions in optimization and variational inequalities , 2002 .

[14]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[15]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Alfred Auslender,et al.  Existence of optimal solutions and duality results under weak conditions , 2000, Math. Program..

[17]  Daniel Cremers,et al.  A convex relaxation approach for computing minimal partitions , 2009, CVPR.

[18]  Minh N. Do,et al.  A revisit to cost aggregation in stereo matching: How far can we reduce its computational redundancy? , 2011, 2011 International Conference on Computer Vision.

[19]  Andreas Weinmann,et al.  The L1-Potts Functional for Robust Jump-Sparse Reconstruction , 2012, SIAM J. Numer. Anal..

[20]  D. Cremers.  Convex Relaxation Techniques for Segmentation , Stereo and Multiview Reconstruction , 2010 .

[21]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[22]  Mila Nikolova,et al.  Description of the Minimizers of Least Squares Regularized with 퓁0-norm. Uniqueness of the Global Minimizer , 2013, SIAM J. Imaging Sci..

[23]  R. B. Potts.  Some generalized order-disorder transformations , 1952, Mathematical Proceedings of the Cambridge Philosophical Society.

[24]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[25]  Jing Yuan,et al.  Discrete Orthogonal Decomposition and Variational Fluid Flow Estimation , 2005, Scale-Space.

[26]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[28]  Arjan Kuijper,et al.  Scale Space and Variational Methods in Computer Vision , 2013, Lecture Notes in Computer Science.

[29]  Pierre-Jean Reissman,et al.  From simulated annealing to stochastic continuation: a new trend in combinatorial optimization , 2012, Journal of Global Optimization.

[30]  Miao Liao,et al.  Real-time Global Stereo Matching Using Hierarchical Belief Propagation , 2006, BMVC.

[31]  Daniel Cremers,et al.  A variational framework for image segmentation combining motion estimation and shape regularization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[32]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[33]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[34]  Joumana Farah,et al.  A Parallel Proximal Splitting Method for Disparity Estimation from Multicomponent Images Under Illumination Variation , 2012, Journal of Mathematical Imaging and Vision.

[35]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[36]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[37]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[38]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[39]  Jing Yuan,et al.  Simultaneous Higher-Order Optical Flow Estimation and Decomposition , 2007, SIAM J. Sci. Comput..

[40]  Ingemar J. Cox,et al.  Dynamic histogram warping of image pairs for constant image brightness , 1995, Proceedings., International Conference on Image Processing.

[41]  Isabelle E. Magnin,et al.  A Stochastic Continuation Approach to Piecewise Constant Reconstruction , 2007, IEEE Transactions on Image Processing.

[42]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[43]  Joachim Weickert,et al.  Why Is the Census Transform Good for Robust Optic Flow Computation? , 2013, SSVM.

[44]  Rachid Deriche,et al.  Optical-Flow Estimation while Preserving Its Discontinuities: A Variational Approach , 1995, ACCV.

[45]  Daniel Cremers,et al.  Near Real-Time Motion Segmentation Using Graph Cuts , 2006, DAGM-Symposium.

[46]  Andreas Weinmann,et al.  Jump-Sparse and Sparse Recovery Using Potts Functionals , 2013, IEEE Transactions on Signal Processing.

[47]  Yvan G. Leclerc,et al.  Constructing simple stable descriptions for image partitioning , 1989, International Journal of Computer Vision.

[48]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[49]  Giuseppe Buttazzo,et al.  General existence theorems for unilateral problems in continuum mechanics , 1988 .

[50]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[51]  Jean-Christophe Pesquet,et al.  Comparison of two proximal splitting algorithms for solving multilabel disparity estimation problems , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[52]  Horst Bischof,et al.  Motion estimation with non-local total variation regularization , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[54]  Otmar Scherzer,et al.  Handbook of Mathematical Methods in Imaging , 2015, Handbook of Mathematical Methods in Imaging.

[55]  Zhaosong Lu,et al.  Iterative hard thresholding methods for l0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} regulari , 2012, Mathematical Programming.

[56]  Jing Yuan,et al.  Convex Hodge Decomposition and Regularization of Image Flows , 2009, Journal of Mathematical Imaging and Vision.

[57]  Christoph Schnörr,et al.  Optical Flow , 2015, Handbook of Mathematical Methods in Imaging.

[58]  Jean-Christophe Pesquet,et al.  Disparity Map Estimation Using A Total Variation Bound , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[59]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Eric L. Miller,et al.  Multiple Hypothesis Video Segmentation from Superpixel Flows , 2010, ECCV.

[61]  O. Scherzer,et al.  ANALYSIS OF OPTICAL FLOW MODELS IN THE FRAMEWORK OF THE CALCULUS OF VARIATIONS , 2002 .

[62]  Hujun Bao,et al.  Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Guanghui Wang,et al.  Stereo matching algorithm based on curvelet decomposition and modified support weights , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[64]  Isabelle E. Magnin,et al.  Optimization by Stochastic Continuation , 2010, SIAM J. Imaging Sci..

[65]  Andreas Weinmann,et al.  Fast Partitioning of Vector-Valued Images , 2014, SIAM J. Imaging Sci..

[66]  G. Winkler,et al.  Complexity Penalized M-Estimation , 2008 .

[67]  Wolfgang Förstner,et al.  Probabilistic Multi-class Scene Flow Segmentation for Traffic Scenes , 2010, DAGM-Symposium.

[68]  Horst Bischof,et al.  Efficient Minimization of the Non-local Potts Model , 2011, SSVM.

[69]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[70]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[71]  Ruigang Yang,et al.  Global stereo matching leveraged by sparse ground control points , 2011, CVPR 2011.

[72]  Michael J. Black,et al.  Layered image motion with explicit occlusions, temporal consistency, and depth ordering , 2010, NIPS.

[73]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[74]  Antonin Chambolle,et al.  Image Segmentation by Variational Methods: Mumford and Shah Functional and the Discrete Approximations , 1995, SIAM J. Appl. Math..

[75]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[76]  Du-Ming Tsai,et al.  The evaluation of normalized cross correlations for defect detection , 2003, Pattern Recognit. Lett..

[77]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.