A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape

Since their introduction as a means of front propagation and their first application to edge-based segmentation in the early 90’s, level set methods have become increasingly popular as a general framework for image segmentation. In this paper, we present a survey of a specific class of region-based level set segmentation methods and clarify how they can all be derived from a common statistical framework.Region-based segmentation schemes aim at partitioning the image domain by progressively fitting statistical models to the intensity, color, texture or motion in each of a set of regions. In contrast to edge-based schemes such as the classical Snakes, region-based methods tend to be less sensitive to noise. For typical images, the respective cost functionals tend to have less local minima which makes them particularly well-suited for local optimization methods such as the level set method.We detail a general statistical formulation for level set segmentation. Subsequently, we clarify how the integration of various low level criteria leads to a set of cost functionals. We point out relations between the different segmentation schemes. In experimental results, we demonstrate how the level set function is driven to partition the image plane into domains of coherent color, texture, dynamic texture or motion. Moreover, the Bayesian formulation allows to introduce prior shape knowledge into the level set method. We briefly review a number of advances in this domain.

[1]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[3]  Jacques-Olivier Lachaud,et al.  Deformable meshes with automated topology changes for coarse-to-fine three-dimensional surface extraction , 1999, Medical Image Anal..

[4]  Daniel Cremers,et al.  Diffusion Snakes: Introducing Statistical Shape Knowledge into the Mumford-Shah Functional , 2002, International Journal of Computer Vision.

[5]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[6]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  W. A. Perkins,et al.  Area Segmentation of Images Using Edge Points , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Daniel Cremers,et al.  Dynamical statistical shape priors for level set-based tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Christoph Schnörr,et al.  Computation of discontinuous optical flow by domain decomposition and shape optimization , 1992, International Journal of Computer Vision.

[10]  E. Ising Beitrag zur Theorie des Ferromagnetismus , 1925 .

[11]  Nikos Paragios,et al.  Shape Priors for Level Set Representations , 2002, ECCV.

[12]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[14]  T. Chan,et al.  A Variational Level Set Approach to Multiphase Motion , 1996 .

[15]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[16]  Song-Chun Zhu,et al.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998, International Journal of Computer Vision.

[17]  Michel Barlaud,et al.  DREAM2S: Deformable Regions Driven by an Eulerian Accurate Minimization Method for Image and Video Segmentation , 2002, ECCV.

[18]  Johan Montagnat,et al.  New Algorithms for Controlling Active Contours Shape and Topology , 2000, ECCV.

[19]  Martin S. Kochmanski NOTE ON THE E. ISING'S PAPER ,,BEITRAG ZUR THEORIE DES FERROMAGNETISMUS" (Zs. Physik, 31, 253 (1925)) , 2008 .

[20]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[21]  Yvan G. Leclerc,et al.  Constructing simple stable descriptions for image partitioning , 1989, International Journal of Computer Vision.

[22]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[23]  Rachid Deriche,et al.  Diffusion tensor regularization with constraints preservation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[24]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[25]  Thomas Brox,et al.  A TV Flow Based Local Scale Measure for Texture Discrimination , 2004, ECCV.

[26]  A. Dervieux,et al.  Multifluid incompressible flows by a finite element method , 1981 .

[27]  Laurent D. Cohen,et al.  Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[29]  Rachid Deriche,et al.  Level Set and Region Based Surface Propagation for Diffusion Tensor MRI Segmentation , 2004, ECCV Workshops CVAMIA and MMBIA.

[30]  W. Eric L. Grimson,et al.  A shape-based approach to the segmentation of medical imagery using level sets , 2003, IEEE Transactions on Medical Imaging.

[31]  Anthony J. Yezzi,et al.  Vessel Segmentation Using a Shape Driven Flow , 2004, MICCAI.

[32]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[33]  Christoph Schnörr,et al.  Natural Image Statistics for Natural Image Segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[34]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[35]  S. Osher,et al.  Algorithms Based on Hamilton-Jacobi Formulations , 1988 .

[36]  W. Eric L. Grimson,et al.  Model-based curve evolution technique for image segmentation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[37]  Rachid Deriche,et al.  Active unsupervised texture segmentation on a diffusion based feature space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[38]  Rachid Deriche,et al.  Geodesic Active Regions: A New Framework to Deal with Frame Partition Problems in Computer Vision , 2002, J. Vis. Commun. Image Represent..

[39]  Philippe Réfrégier,et al.  Influence of the noise model on level set active contour segmentation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Olivier Faugeras,et al.  Approximations of Shape Metrics and Application to Shape Warping and Empirical Shape Statistics , 2006, Statistics and Analysis of Shapes.

[41]  Daniel Cremers,et al.  Nonlinear Shape Statistics in Mumford-Shah Based Segmentation , 2002, ECCV.

[42]  Luminita A. Vese,et al.  Multiphase Object Detection and Image Segmentation , 2003 .

[43]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[44]  Anthony J. Yezzi,et al.  A statistical approach to snakes for bimodal and trimodal imagery , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[45]  Rachid Deriche,et al.  Tensor Processing for Texture and Colour Segmentation , 2005, SCIA.

[46]  Daniel Cremers,et al.  A Generative Model Based Approach to Motion Segmentation , 2003, DAGM-Symposium.

[47]  B. De Moor,et al.  Subspace angles between linear stochastic models , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[48]  J. Bigun,et al.  Optimal Orientation Detection of Linear Symmetry , 1987, ICCV 1987.

[49]  Daniel Cremers,et al.  Variational space-time motion segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[50]  Josiane Zerubia,et al.  A Variational Model for Image Classification and Restoration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Silvano Di Zenzo,et al.  A note on the gradient of a multi-image , 1986, Comput. Vis. Graph. Image Process..

[52]  A. Ravishankar Rao,et al.  Computing oriented texture fields , 1991, CVGIP Graph. Model. Image Process..

[53]  S. Osher,et al.  Geometric Level Set Methods in Imaging, Vision, and Graphics , 2011, Springer New York.

[54]  Daniel Cremers,et al.  Dynamic texture segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[55]  Daniel Cremers,et al.  Efficient Kernel Density Estimation of Shape and Intensity Priors for Level Set Segmentation , 2005, MICCAI.

[56]  M. Barlaud,et al.  Shape gradient for multimodal image segmentation using mutual information , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[57]  Baba C. Vemuri,et al.  Topology-independent shape modeling scheme , 1993, Optics & Photonics.

[58]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Rachid Deriche,et al.  Implicit Active Shape Models for 3D Segmentation in MR Imaging , 2004, MICCAI.

[60]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[61]  Anthony J. Yezzi,et al.  Information-Theoretic Active Polygons for Unsupervised Texture Segmentation , 2005, International Journal of Computer Vision.

[62]  F. Ghoreishi,et al.  The Tau method and a new preconditioner , 2004 .

[63]  Zhizhou Wang,et al.  An affine invariant tensor dissimilarity measure and its applications to tensor-valued image segmentation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[64]  J. A. Dieudonne SHAPE GRADIENT FOR MULTI-MODAL IMAGE SEGMENTATION USING MUTUAL INFORMATION , 2004 .

[65]  Tony F. Chan,et al.  A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model , 2002, International Journal of Computer Vision.

[66]  Rachid Deriche,et al.  Unsupervised Segmentation Incorporating Colour, Texture, and Motion , 2003, CAIP.

[67]  Daniel Cremers,et al.  Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation , 2006, International Journal of Computer Vision.

[68]  O. Faugeras,et al.  Statistics on Multivariate Normal Distributions: A Geometric Approach and its Application to Diffusion Tensor MRI , 2004 .

[69]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[70]  V. Caselles,et al.  A geometric model for active contours in image processing , 1993 .

[71]  Rachid Deriche,et al.  Segmentation of 3D Probability Density Fields by Surface Evolution: Application to Diffusion MRI , 2004, MICCAI.

[72]  Baba C. Vemuri,et al.  Front Propagation: A Framework for Topology Independent Shape Modeling and Recovery , 1994 .

[73]  Stefano Soatto,et al.  Shape representation via harmonic embedding , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[74]  John W. Fisher,et al.  Nonparametric methods for image segmentation using information theory and curve evolution , 2002, Proceedings. International Conference on Image Processing.

[75]  DericheRachid,et al.  A Review of Statistical Approaches to Level Set Segmentation , 2007 .

[76]  Anthony J. Yezzi,et al.  Gradient flows and geometric active contour models , 1995, Proceedings of IEEE International Conference on Computer Vision.

[77]  Jan Sokolowski,et al.  Introduction to shape optimization , 1992 .

[78]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[79]  Thomas Brox,et al.  Variational Motion Segmentation with Level Sets , 2006, ECCV.

[80]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[81]  Daniel Cremers,et al.  Kernel Density Estimation and Intrinsic Alignment for Knowledge-Driven Segmentation: Teaching Level Sets to Walk , 2004, DAGM-Symposium.

[82]  M. Hassner,et al.  The use of Markov Random Fields as models of texture , 1980 .

[83]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[84]  O. Faugeras,et al.  Statistical shape influence in geodesic active contours , 2002, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[85]  Daniel Cremers,et al.  A Multiphase Dynamic Labeling Model for Variational Recognition-driven Image Segmentation , 2005, International Journal of Computer Vision.

[86]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[87]  A. Dervieux,et al.  A finite element method for the simulation of a Rayleigh-Taylor instability , 1980 .

[88]  R. Deriche,et al.  A variational framework for active and adaptative segmentation of vector valued images , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[89]  Amar Mitiche,et al.  Spatio-temporal motion segmentation via level set partial differential equations , 2002, Proceedings Fifth IEEE Southwest Symposium on Image Analysis and Interpretation.

[90]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[91]  Thomas Brox,et al.  Level Set Based Image Segmentation with Multiple Regions , 2004, DAGM-Symposium.

[92]  Olivier D. Faugeras,et al.  Image Segmentation Using Active Contours: Calculus of Variations or Shape Gradients? , 2003, SIAM J. Appl. Math..

[93]  Daniel Cremers,et al.  Binary Partitioning, Perceptual Grouping, and Restoration with Semidefinite Programming , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[94]  Daniel Cremers,et al.  A variational framework for image segmentation combining motion estimation and shape regularization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[95]  Rachid Deriche,et al.  Geodesic active regions and level set methods for motion estimation and tracking , 2005, Comput. Vis. Image Underst..

[96]  MumfordDavid,et al.  Filters, Random Fields and Maximum Entropy (FRAME) , 1998 .

[97]  Demetri Terzopoulos,et al.  Topologically adaptable snakes , 1995, Proceedings of IEEE International Conference on Computer Vision.

[98]  Daniel Cremers,et al.  Motion Competition: Variational Integration of Motion Segmentation and Shape Regularization , 2002, DAGM-Symposium.

[99]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[100]  L. Skovgaard A Riemannian geometry of the multivariate normal model , 1984 .

[101]  Thomas S. Huang,et al.  Image processing , 1971 .

[102]  Daniel Cremers,et al.  Motion Competition: A variational framework for piecewise parametric motion segmentation , 2005 .

[103]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.

[104]  Nahum Kiryati,et al.  Dense discontinuous optical flow via contour-based segmentation , 2005, IEEE International Conference on Image Processing 2005.

[105]  Josiane Zerubia,et al.  Higher Order Active Contours , 2006, International Journal of Computer Vision.

[106]  R. Deriche,et al.  Regularization of orthonormal vector sets using coupled PDE's , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[107]  Anthony J. Yezzi,et al.  Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification , 2001, IEEE Trans. Image Process..

[108]  Yunmei Chen,et al.  Using Prior Shapes in Geometric Active Contours in a Variational Framework , 2002, International Journal of Computer Vision.

[109]  Ulf Grenander,et al.  Hands: A Pattern Theoretic Study of Biological Shapes , 1990 .

[110]  S. Mallat Multiresolution approximations and wavelet orthonormal bases of L^2(R) , 1989 .

[111]  T. Brox,et al.  Diffusion and regularization of vector- and matrix-valued images , 2002 .

[112]  Johan Wiklund,et al.  Multidimensional orientation : texture analysis and optical flow , 1991 .