Variational image segmentation model coupled with image restoration achievements

Image segmentation and image restoration are two important topics in image processing with a number of important applications. In this paper, we propose a new multiphase segmentation model by combining image restoration and image segmentation models. Utilizing aspects of image restoration, the proposed segmentation model can effectively and robustly tackle images with a high level of noise or blurriness, missing pixels or vector values. In particular, one of the most important segmentation models, the piecewise constant Mumford-Shah model, can be extended easily in this way to segment gray and vector-valued images corrupted, for example, by noise, blur or information loss after coupling a new data fidelity term which borrowed from the field of image restoration. It can be solved efficiently using the alternating minimization algorithm, and we prove the convergence of this algorithm with three variables under mild conditions. Experiments on many synthetic and real-world images demonstrate that our method gives better segmentation results in terms of quality and quantity in comparison to other state-of-the-art segmentation models, especially for blurry images and those with information loss. HighlightsProvide a new methodology and a new model for multiphase image segmentation.Extend the piecewise constant Mumford-Shah model to handle blurry case easily.Has the ability to segment images with Gaussian, Poisson, and impulsive noises.Effective in segmenting vector-valued image and image observed with information loss.The convergence of the AM algorithm with three variables adopted is proved.

[1]  Mostafa Kaveh,et al.  Fourth-order partial differential equations for noise removal , 2000, IEEE Trans. Image Process..

[2]  Michael K. Ng,et al.  Fast Image Restoration Methods for Impulse and Gaussian Noises Removal , 2009, IEEE Signal Processing Letters.

[3]  J. Morel,et al.  A multiscale algorithm for image segmentation by variational method , 1994 .

[4]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[5]  Raymond H. Chan,et al.  Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization , 2005, IEEE Transactions on Image Processing.

[6]  Gabriele Steidl,et al.  A new fuzzy c-means method with total variation regularization for segmentation of images with noisy and incomplete data , 2012, Pattern Recognit..

[7]  Tony F. Chan,et al.  Mumford and Shah Model and Its Applications to Image Segmentation and Image Restoration , 2015, Handbook of Mathematical Methods in Imaging.

[8]  Thomas Brox,et al.  On the Equivalence of Soft Wavelet Shrinkage, Total Variation Diffusion, Total Variation Regularization, and SIDEs , 2004, SIAM J. Numer. Anal..

[9]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[10]  Daniel Cremers,et al.  A convex relaxation approach for computing minimal partitions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[12]  Stanley Osher,et al.  Total variation based image restoration with free local constraints , 1994, Proceedings of 1st International Conference on Image Processing.

[13]  Raymond H. Chan,et al.  A Two-Stage Image Segmentation Method Using a Convex Variant of the Mumford-Shah Model and Thresholding , 2013, SIAM J. Imaging Sci..

[14]  Ivo F. Sbalzarini,et al.  Coupling Image Restoration and Segmentation: A Generalized Linear Model/Bregman Perspective , 2013, International Journal of Computer Vision.

[15]  Gabriele Steidl,et al.  Multiclass Segmentation by Iterated ROF Thresholding , 2013, EMMCVPR.

[16]  A. Chambolle FINITE-DIFFERENCES DISCRETIZATIONS OF THE MUMFORD-SHAH FUNCTIONAL , 1999 .

[17]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[18]  Hyenkyun Woo,et al.  Linearized proximal alternating minimization algorithm for motion deblurring by nonlocal regularization , 2011, Pattern Recognit..

[19]  Thomas J. Asaki,et al.  A Variational Approach to Reconstructing Images Corrupted by Poisson Noise , 2007, Journal of Mathematical Imaging and Vision.

[20]  E. Giusti Minimal surfaces and functions of bounded variation , 1977 .

[21]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[22]  Antonin Chambolle,et al.  Image Segmentation by Variational Methods: Mumford and Shah Functional and the Discrete Approximations , 1995, SIAM J. Appl. Math..

[23]  Massimo Gobbino Finite Difference Approximation of the Mumford-Shah Functional , 1998 .

[24]  I. Csiszár Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[25]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[26]  Chen Hu,et al.  An iterative procedure for removing random-valued impulse noise , 2004, IEEE Signal Processing Letters.

[27]  Mila Nikolova,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004, Journal of Mathematical Imaging and Vision.

[28]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[29]  Arvid Lundervold,et al.  Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time , 2003, IEEE Trans. Image Process..

[30]  Jian-Feng Cai,et al.  A framelet-based image inpainting algorithm , 2008 .

[31]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[32]  Tony F. Chan,et al.  High-Order Total Variation-Based Image Restoration , 2000, SIAM J. Sci. Comput..

[33]  Y. V. Venkatesh,et al.  Deblurring Gaussian blur using a wavelet array transform , 1995, Pattern Recognit..

[34]  Gilles Aubert,et al.  A Variational Approach to Removing Multiplicative Noise , 2008, SIAM J. Appl. Math..

[35]  C. W. Groetsch,et al.  Regularization of Ill-Posed Problems. , 1978 .

[36]  R. B. Potts Some generalized order-disorder transformations , 1952, Mathematical Proceedings of the Cambridge Philosophical Society.

[37]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[38]  Xue-Cheng Tai,et al.  A Continuous Max-Flow Approach to Potts Model , 2010, ECCV.

[39]  Tony F. Chan,et al.  A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model , 2002, International Journal of Computer Vision.

[40]  Gabriele Steidl,et al.  Removing Multiplicative Noise by Douglas-Rachford Splitting Methods , 2010, Journal of Mathematical Imaging and Vision.

[41]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[42]  A. N. Tikhonov,et al.  The regularization of ill-posed problems , 1963 .

[43]  W. Fleming,et al.  An integral formula for total gradient variation , 1960 .

[44]  G. Steidl,et al.  Variational Methods with Higher–Order Derivatives in Image Processing , 2007 .

[45]  Michael K. Ng,et al.  A Multiphase Image Segmentation Method Based on Fuzzy Region Competition , 2010, SIAM J. Imaging Sci..

[46]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[47]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .