Pairwise Similarities across Images for Multiple View Rigid/Non-Rigid Segmentation and Registration

A variational approach for the background/foreground segmentation of multiple views of the same scene is presented. The main novelty is the introduction of cost functions based on pairwise similarity between pixels across different images. These cost functions are minimized within a level set framework. In addition, a warping model (rigid or non-rigid) between the emerging foregrounds in the different views is imposed, thus avoiding the introduction of a specific shape term in the cost function to handle occlusions. The thin plate spline (TPS) warping is for the first time employed within the level set framework to model non- rigid deformations. The minimization of these cost functions leads to simultaneous segmentation and registration of the different views. Examples of segmentations of a variety of objects are shown and possible applications are proposed.

[1]  Anthony J. Yezzi,et al.  A statistical approach to snakes for bimodal and trimodal imagery , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Ming-Hsuan Yang,et al.  A direct method for modeling non-rigid motion with thin plate spline , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[4]  Stefano Soatto,et al.  Local Features, All Grown Up , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  J. Sethian,et al.  FRONTS PROPAGATING WITH CURVATURE DEPENDENT SPEED: ALGORITHMS BASED ON HAMILTON-JACOB1 FORMULATIONS , 2003 .

[6]  W. Eric L. Grimson,et al.  A shape-based approach to the segmentation of medical imagery using level sets , 2003, IEEE Transactions on Medical Imaging.

[7]  Anthony J. Yezzi,et al.  A variational framework for integrating segmentation and registration through active contours , 2003, Medical Image Anal..

[8]  Stefano Soatto,et al.  Region matching with missing parts , 2002, Image Vis. Comput..

[9]  Gregory G. Slabaugh,et al.  Coupled PDEs for non-rigid registration and segmentation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Daniel Cremers,et al.  Multiphase Dynamic Labeling for Variational Recognition-Driven Image Segmentation , 2004, ECCV.

[12]  Tony F. Chan,et al.  Level set based shape prior segmentation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  S. Osher,et al.  Level set methods: an overview and some recent results , 2001 .

[15]  Anthony J. Yezzi,et al.  Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification , 2001, IEEE Trans. Image Process..

[16]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[17]  Nahum Kiryati,et al.  Prior-based Segmentation and Shape Registration in the Presence of Perspective Distortion , 2007, International Journal of Computer Vision.

[18]  Nahum Kiryati,et al.  Mutual Segmentation with Level Sets , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[19]  Yunmei Chen,et al.  Using Prior Shapes in Geometric Active Contours in a Variational Framework , 2002, International Journal of Computer Vision.

[20]  B. S. Manjunath,et al.  Graph partitioning active contours (GPAC) for image segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[22]  Daniel Cremers,et al.  Dynamical statistical shape priors for level set-based tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Nikos Paragios,et al.  Shape Priors for Level Set Representations , 2002, ECCV.

[24]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[25]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[26]  Nahum Kiryati,et al.  Unlevel-Sets: Geometry and Prior-Based Segmentation , 2004, ECCV.

[27]  Nahum Kiryati,et al.  Prior-based segmentation by projective registration and level sets , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[28]  Olivier D. Faugeras,et al.  Statistical shape influence in geodesic active contours , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[29]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[30]  Daniel Cremers,et al.  Diffusion Snakes: Introducing Statistical Shape Knowledge into the Mumford-Shah Functional , 2002, International Journal of Computer Vision.