Extraction of digital wavefront sets using applied harmonic analysis and deep neural networks

Microlocal analysis provides deep insight into singularity structures and is often crucial for solving inverse problems, predominately, in imaging sciences. Of particular importance is the analysis of wavefront sets and the correct extraction of those. In this paper, we introduce the first algorithmic approach to extract the wavefront set of images, which combines data-based and model-based methods. Based on a celebrated property of the shearlet transform to unravel information on the wavefront set, we extract the wavefront set of an image by first applying a discrete shearlet transform and then feeding local patches of this transform to a deep convolutional neural network trained on labeled data. The resulting algorithm outperforms all competing algorithms in edge-orientation and ramp-orientation detection.

[1]  E. Candès,et al.  Continuous Curvelet Transform : I . Resolution of the Wavefront Set , 2003 .

[2]  D. Labate,et al.  Resolution of the wavefront set using continuous shearlets , 2006, math/0605375.

[3]  Zeljko Kereta,et al.  Continuous Parabolic Molecules , 2016 .

[4]  V. FABER,et al.  Inversion of cone-beam data and helical tomography , 1995 .

[5]  L. Hörmander,et al.  The Analysis of Linear Partial Differential Operators I: Distribution Theory and Fourier Analysis , 1983 .

[6]  Wojciech Samek,et al.  Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography , 2018, Inverse Problems.

[7]  Thomas O. Binford,et al.  Inferring Surfaces from Images , 1981, Artif. Intell..

[8]  Wang-Q Lim,et al.  Edge analysis and identification using the continuous shearlet transform , 2009 .

[9]  Jianbo Shi,et al.  DeepEdge: A multi-scale bifurcated deep network for top-down contour detection , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Alexander Binder,et al.  Object Boundary Detection and Classification with Image-Level Labels , 2017, GCPR.

[11]  L. Hörmander Fourier integral operators. I , 1995 .

[12]  Irwin Sobel,et al.  An Isotropic 3×3 image gradient operator , 1990 .

[13]  Hamid Krim,et al.  A Shearlet Approach to Edge Analysis and Detection , 2009, IEEE Transactions on Image Processing.

[14]  M. Czubak,et al.  PSEUDODIFFERENTIAL OPERATORS , 2020, Introduction to Partial Differential Equations.

[15]  Jitendra Malik,et al.  Detecting and localizing edges composed of steps, peaks and roofs , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[16]  Subhransu Maji,et al.  Semantic contours from inverse detectors , 2011, 2011 International Conference on Computer Vision.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Philipp Grohs,et al.  Continuous shearlet frames and resolution of the wavefront set , 2009, 0909.3712.

[19]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Michael Brady,et al.  Computational Approaches to Image Understanding , 1982, CSUR.

[21]  Gunther Uhlmann,et al.  The inverse problem for the local geodesic ray transform , 2012, 1210.2084.

[22]  Ming-Yu Liu,et al.  CASENet: Deep Category-Aware Semantic Edge Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[24]  Emily J. King,et al.  Shearlet-based detection of flame fronts , 2015, ArXiv.

[25]  E. T. Quinto,et al.  Local Tomography in Electron Microscopy , 2008, SIAM J. Appl. Math..

[26]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[27]  Eric Todd Quinto,et al.  Singularities of the X-ray transform and limited data tomography , 1993 .

[28]  Eric Todd Quinto,et al.  Microlocal Analysis in Tomography , 2015, Handbook of Mathematical Methods in Imaging.

[29]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[30]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Laurent Jacques,et al.  A panorama on multiscale geometric representations, intertwining spatial, directional and frequency selectivity , 2011, Signal Process..

[34]  Zhuowen Tu,et al.  Supervised Learning of Edges and Object Boundaries , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[35]  G. Folland Introduction to Partial Differential Equations , 1976 .

[36]  Laurent Demanet,et al.  Fast Computation of Fourier Integral Operators , 2006, SIAM J. Sci. Comput..

[37]  E. Candès,et al.  Continuous curvelet transform , 2003 .

[38]  M. E. Davison,et al.  The Ill-Conditioned Nature of the Limited Angle Tomography Problem , 1983 .

[39]  G. Kutyniok,et al.  Digital Shearlet Transform , 2011, ArXiv.

[40]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[41]  G. Kutyniok,et al.  Classification of Edges Using Compactly Supported Shearlets , 2014, 1411.5657.

[42]  R. Marks Introduction to Shannon Sampling and Interpolation Theory , 1990 .

[43]  D. Labate,et al.  Sparse Multidimensional Representations using Anisotropic Dilation and Shear Operators , 2006 .

[44]  Hartmut Führ,et al.  Resolution of the Wavefront Set Using General Continuous Wavelet Transforms , 2014, 1412.7158.

[45]  Wang-Q Lim,et al.  ShearLab 3D , 2014, 1402.5670.

[46]  Tomaso A. Poggio,et al.  On Edge Detection , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Yang Zou,et al.  Simultaneous Edge Alignment and Learning , 2018, ECCV.

[48]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.