Robust Simultaneous Registration and Segmentation with Sparse Error Reconstruction

We introduce a fast and efficient variational framework for Simultaneous Registration and Segmentation (SRS) applicable to a wide variety of image sequences. We demonstrate that a dense correspondence map (between consecutive frames) can be reconstructed correctly even in the presence of partial occlusion, shading, and reflections. The errors are efficiently handled by exploiting their sparse nature. In addition, the segmentation functional is reformulated using a dual Rudin-Osher-Fatemi (ROF) model for fast implementation. Moreover, nonparametric shape prior terms that are suited for this dual-ROF model are proposed. The efficacy of the proposed method is validated with extensive experiments on both indoor, outdoor natural and biological image sequences, demonstrating the higher accuracy and efficiency compared to various state-of-the-art methods.

[1]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[2]  Daniel Cremers,et al.  Dynamical statistical shape priors for level set-based tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[4]  Li Bai,et al.  Minimum error bounded efficient ℓ1 tracker with occlusion detection , 2011, CVPR 2011.

[5]  Nikos Paragios,et al.  Registration with Uncertainties and Statistical Modeling of Shapes with Variable Metric Kernels , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[7]  Yunmei Chen,et al.  A Variational PDE Based Level Set Method for a Simultaneous Segmentation and Non-rigid Registration , 2005, MICCAI.

[8]  Milan Sonka,et al.  Multistage hybrid active appearance model matching: segmentation of left and right ventricles in cardiac MR images , 2001, IEEE Transactions on Medical Imaging.

[9]  Haibin Ling,et al.  Robust visual tracking using ℓ1 minimization , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[11]  Daniel Cremers,et al.  Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation , 2006, International Journal of Computer Vision.

[12]  James S. Duncan,et al.  Segmentation of left ventricle from 3D cardiac MR image sequences using a subject-specific dynamical model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[14]  Daniel Cremers,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 a Combinatorial Solution for Model-based Image Segmentation and Real-time Tracking , 2022 .

[15]  Farida Cheriet,et al.  Region-Based Segmentation via Non-Rigid Template Matching , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[17]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[18]  Xavier Bresson,et al.  A Variational Model for Object Segmentation Using Boundary Information and Shape Prior Driven by the Mumford-Shah Functional , 2006, International Journal of Computer Vision.

[19]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[20]  Ming-Hsuan Yang,et al.  Visual tracking with online Multiple Instance Learning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Daniel Cremers,et al.  High resolution motion layer decomposition using dual-space graph cuts , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[23]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[24]  Pratim Ghosh,et al.  Generalized simultaneous registration and segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Ian H. Jermyn,et al.  Region Extraction from Multiple Images , 2001, ICCV.

[27]  Xavier Bresson,et al.  Fast Global Minimization of the Active Contour/Snake Model , 2007, Journal of Mathematical Imaging and Vision.

[28]  Daniel Cremers,et al.  Globally optimal shape-based tracking in real-time , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Ming-Hsuan Yang,et al.  Incremental Learning for Robust Visual Tracking , 2008, International Journal of Computer Vision.

[30]  Heinz Handels,et al.  A Variational Approach for Combined Segmentation and Estimation of Respiratory Motion in Temporal Image Sequences , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[31]  Daniel Cremers,et al.  An Improved Algorithm for TV-L 1 Optical Flow , 2009, Statistical and Geometrical Approaches to Visual Motion Analysis.

[32]  Gregory G. Slabaugh,et al.  Coupled PDEs for non-rigid registration and segmentation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Stefano Soatto,et al.  A Pseudo-distance for Shape Priors in Level Set Segmentation , 2003 .

[34]  Olivier D. Faugeras,et al.  Statistical shape influence in geodesic active contours , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[35]  Thomas Brox,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Highly Accurate Optic Flow Computation with Theoretically Justified Warping Highly Accurate Optic Flow Computation with Theoretically Justified Warping , 2022 .

[36]  W. Eric L. Grimson,et al.  A shape-based approach to the segmentation of medical imagery using level sets , 2003, IEEE Transactions on Medical Imaging.

[37]  Pratim Ghosh,et al.  Robust dynamical model for simultaneous registration and segmentation in a variational framework: A Bayesian approach , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[38]  Shai Avidan,et al.  Ensemble Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..