Wavelet-Based Segmentation on the Sphere

Abstract Segmentation, a useful/powerful technique in pattern recognition, is the process of identifying object outlines within images. There are a number of efficient algorithms for segmentation in Euclidean space that depend on the variational approach and partial differential equation modelling. Wavelets have been used successfully in various problems in image processing, including segmentation, inpainting, noise removal, super-resolution image restoration, and many others. Wavelets on the sphere have been developed to solve such problems for data defined on the sphere, which arise in numerous fields such as cosmology and geophysics. In this work, we propose a wavelet-based method to segment images on the sphere, accounting for the underlying geometry of spherical data. Our method is a direct extension of the tight-frame based segmentation method used to automatically identify tube-like structures such as blood vessels in medical imaging. It is compatible with any arbitrary type of wavelet frame defined on the sphere, such as axisymmetric wavelets, directional wavelets, curvelets, and hybrid wavelet constructions. Such an approach allows the desirable properties of wavelets to be naturally inherited in the segmentation process. In particular, directional wavelets and curvelets, which were designed to efficiently capture directional signal content, provide additional advantages in segmenting images containing prominent directional and curvilinear features. We present several numerical experiments, applying our wavelet-based segmentation method, as well as the common K-means method, on real-world spherical images, including an Earth topographic map, a light probe image, solar data-sets, and spherical retina images. These experiments demonstrate the superiority of our method and show that it is capable of segmenting different kinds of spherical images, including those with prominent directional features. Moreover, our algorithm is efficient with convergence usually within a few iterations.

[1]  Belgium,et al.  Correspondence principle between spherical and euclidean wavelets , 2005, astro-ph/0502486.

[2]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[3]  Andreas Weinmann,et al.  Fast Partitioning of Vector-Valued Images , 2014, SIAM J. Imaging Sci..

[4]  M. P. Hobson,et al.  Cosmological Applications of a Wavelet Analysis on the Sphere , 2007, 0704.3158.

[5]  Raymond H. Chan,et al.  A Two-Stage Image Segmentation Method Using a Convex Variant of the Mumford-Shah Model and Thresholding , 2013, SIAM J. Imaging Sci..

[6]  Max Welling,et al.  Spherical CNNs , 2018, ICLR.

[7]  Kostas Daniilidis,et al.  Normalized Cross-Correlation for Spherical Images , 2004, ECCV.

[8]  Domenico Marinucci,et al.  Mixed Needlets , 2010, 1006.3835.

[9]  Raymond H. Chan,et al.  Vessel Segmentation in Medical Imaging Using a Tight-Frame-Based Algorithm , 2011, SIAM J. Imaging Sci..

[10]  Mila Nikolova,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004, Journal of Mathematical Imaging and Vision.

[11]  Michael P. Hobson,et al.  Fast Directional Continuous Spherical Wavelet Transform Algorithms , 2005, IEEE Transactions on Signal Processing.

[12]  Jian-Feng Cai,et al.  A framelet-based image inpainting algorithm , 2008 .

[13]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[14]  J. D. McEwen,et al.  Data compression on the sphere , 2011, 1108.3900.

[15]  Jean-Luc Starck,et al.  Multichannel Poisson denoising and deconvolution on the sphere: application to the Fermi Gamma-ray Space Telescope , 2012, 1206.2787.

[16]  Pierre Vandergheynst,et al.  Wavelets on the n-sphere and related manifolds , 1998 .

[17]  Yves Wiaux,et al.  A Novel Sampling Theorem on the Sphere , 2011, IEEE Transactions on Signal Processing.

[18]  Pierre Vandergheynst,et al.  On spin scale-discretised wavelets on the sphere for the analysis of CMB polarisation , 2014, Proceedings of the International Astronomical Union.

[19]  P. Vandergheynst,et al.  Wavelets on the 2-sphere: A group-theoretical approach , 1999 .

[20]  O. Blanc,et al.  Exact reconstruction with directional wavelets on the sphere , 2007, 0712.3519.

[21]  Guust Nolet,et al.  Solving or resolving global tomographic models with spherical wavelets, and the scale and sparsity of seismic heterogeneity , 2011 .

[22]  Laurent Jacques,et al.  Wavelets on the sphere: implementation and approximations , 2002 .

[23]  Yves Wiaux,et al.  Sparse Image Reconstruction on the Sphere: Analysis and Synthesis , 2016, IEEE Transactions on Image Processing.

[24]  S. Voronin,et al.  Solving or resolving global tomographic models with spherical wavelets, and the scale and sparsity of seismic heterogeneity , 2010, 1104.3151.

[25]  Olivier D. Faugeras,et al.  CURVES: Curve evolution for vessel segmentation , 2001, Medical Image Anal..

[26]  Michael K. Ng,et al.  A Multiphase Image Segmentation Method Based on Fuzzy Region Competition , 2010, SIAM J. Imaging Sci..

[27]  E. Candès,et al.  Continuous curvelet transform: II. Discretization and frames , 2005 .

[28]  Raymond H. Chan,et al.  A Three-Stage Approach for Segmenting Degraded Color Images: Smoothing, Lifting and Thresholding (SLaT) , 2015, J. Sci. Comput..

[29]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Kostas Daniilidis,et al.  Learning SO(3) Equivariant Representations with Spherical CNNs , 2017, International Journal of Computer Vision.

[31]  Pierre Vandergheynst,et al.  S2LET: A code to perform fast wavelet analysis on the sphere , 2012, ArXiv.

[32]  E. Candès,et al.  Continuous curvelet transform , 2003 .

[33]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[34]  Shigang Li,et al.  A Full-View Spherical Image Format , 2010, 2010 20th International Conference on Pattern Recognition.

[35]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[36]  Yves Wiaux,et al.  Localisation of directional scale-discretised wavelets on the sphere , 2015, ArXiv.

[37]  Raymond H. Chan,et al.  Linkage between Piecewise Constant Mumford-Shah model and ROF model and its virtue in image segmentation , 2018, ArXiv.

[38]  Nahum Kiryati,et al.  On Symmetry, Perspectivity, and Level-Set-Based Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Tony F. Chan,et al.  Active Contours without Edges for Vector-Valued Images , 2000, J. Vis. Commun. Image Represent..

[40]  Matthias Nießner,et al.  Spherical CNNs on Unstructured Grids , 2019, ICLR.

[41]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  J. D. McEwen,et al.  A high-significance detection of non-Gaussianity in the WMAP 5-yr data using directional spherical wavelets , 2006, 0803.2157.

[43]  Peter Schröder,et al.  Spherical wavelets: efficiently representing functions on the sphere , 1995, SIGGRAPH.

[44]  Jason D. McEwen,et al.  Ridgelet transform on the sphere , 2015, ArXiv.

[45]  Paolo Baldi,et al.  Spherical needlets for cosmic microwave background data analysis , 2008 .

[46]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Nathanael Perraudin,et al.  DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications , 2018, Astron. Comput..

[48]  Krzysztof M. Gorski,et al.  NeedATool: A NEEDLET ANALYSIS TOOL FOR COSMOLOGICAL DATA PROCESSING , 2010, 1010.1371.

[49]  J. D. McEwen,et al.  Detection of the ISW effect and corresponding dark energy constraints , 2006 .

[50]  A. M. M. Scaife,et al.  Simulating full‐sky interferometric observations , 2008, 0803.2165.

[51]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[52]  Anthony N. Lasenby,et al.  Testing the Gaussianity of the COBE DMR data with spherical wavelets , 2000 .

[53]  Herbert F Jelinek,et al.  Automated segmentation of retinal blood vessels and identification of proliferative diabetic retinopathy. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[54]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[55]  Andreas Geiger,et al.  SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images , 2018, ECCV.

[56]  Michael P. Hobson,et al.  A directional continuous wavelet transform on the sphere , 2006, ArXiv.

[57]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[58]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  P. Baldi,et al.  Asymptotics for spherical needlets , 2006, math/0606599.

[60]  Qing Wang,et al.  Fast and robust segmentation of spherical particles in volumetric data sets from brightfield microscopy , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[61]  Yves Wiaux,et al.  A Novel Sampling Theorem on the Rotation Group , 2015, IEEE Signal Processing Letters.

[62]  Xiaohao Cai,et al.  Variational image segmentation model coupled with image restoration achievements , 2014, Pattern Recognit..

[63]  Yves Wiaux,et al.  Directional spin wavelets on the sphere , 2015, ArXiv.

[64]  Jason D. McEwen,et al.  Second-Generation Curvelets on the Sphere , 2015, IEEE Transactions on Signal Processing.

[65]  Aaron F. Bobick,et al.  Multiscale 3-D Shape Representation and Segmentation Using Spherical Wavelets , 2007, IEEE Transactions on Medical Imaging.

[66]  E. Candès,et al.  Continuous Curvelet Transform : I . Resolution of the Wavefront Set , 2003 .

[67]  A. Ron,et al.  Affine Systems inL2(Rd): The Analysis of the Analysis Operator , 1997 .

[68]  Pascal Audet,et al.  Toward mapping the effective elastic thickness of planetary lithospheres from a spherical wavelet analysis of gravity and topography , 2014 .

[69]  Arivazhagan Selvaraj,et al.  Texture segmentation using wavelet transform , 2003, Pattern Recognit. Lett..

[70]  Tony F. Chan,et al.  Mumford and Shah Model and Its Applications to Image Segmentation and Image Restoration , 2015, Handbook of Mathematical Methods in Imaging.

[71]  Michael Unser,et al.  Texture classification and segmentation using wavelet frames , 1995, IEEE Trans. Image Process..

[72]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[73]  Serena Morigi,et al.  Segmentation of 3D Tubular Structures by a PDE-Based Anisotropic Diffusion Model , 2008, MMCS.

[74]  Gabriele Steidl,et al.  Multiclass Segmentation by Iterated ROF Thresholding , 2013, EMMCVPR.

[75]  Jean-Luc Starck,et al.  Wavelets, ridgelets and curvelets on the sphere , 2006 .

[76]  Jean-Philippe Thiran,et al.  Sparse Image Reconstruction on the Sphere: Implications of a New Sampling Theorem , 2012, IEEE Transactions on Image Processing.

[77]  Elizabeth L. Shoenfelt.  101 Teambuilding Activities: Ideas Every Coach Can Use to Enhance Teamwork, Communication and Trust , 2005 .

[78]  Marcos López-Caniego,et al.  Wavelets on the sphere. Application to the detection problem , 2006, 2006 14th European Signal Processing Conference.

[79]  Pencho Petrushev,et al.  Localized Tight Frames on Spheres , 2006, SIAM J. Math. Anal..

[80]  Raymond H. Chan,et al.  A Two-Stage Image Segmentation Method for Blurry Images with Poisson or Multiplicative Gamma Noise , 2014, SIAM J. Imaging Sci..

[81]  Donald A Adjeroh,et al.  Texton-based segmentation of retinal vessels. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[82]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[83]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[84]  Yogesh Rathi,et al.  On approximation of orientation distributions by means of spherical ridgelets , 2008, ISBI.

[85]  Pierre Vandergheynst,et al.  On the computation of directional scale-discretized wavelet transforms on the sphere , 2013, Optics & Photonics - Optical Engineering + Applications.

[86]  Martin Greiner,et al.  Wavelets , 2018, Complex..

[87]  Raymond H. Chan,et al.  Framelet-Based Algorithm for Segmentation of Tubular Structures , 2011, SSVM.

[88]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[89]  Carl-Fredrik Westin,et al.  Sparse Multi-Shell Diffusion Imaging , 2011, MICCAI.