Efficient Spatially Adaptive Convolution and Correlation

Fast methods for convolution and correlation underlie a variety of applications in computer vision and graphics, including efficient filtering, analysis, and simulation. However, standard convolution and correlation are inherently limited to fixed filters: spatial adaptation is impossible without sacrificing efficient computation. In early work, Freeman and Adelson have shown how steerable filters can address this limitation, providing a way for rotating the filter as it is passed over the signal. In this work, we provide a general, representation-theoretic, framework that allows for spatially varying linear transformations to be applied to the filter. This framework allows for efficient implementation of extended convolution and correlation for transformation groups such as rotation (in 2D and 3D) and scale, and provides a new interpretation for previous methods including steerable filters and the generalized Hough transform. We present applications to pattern matching, image feature description, vector field visualization, and adaptive image filtering.

[1]  Yacov Hel-Or,et al.  Design of multi-parameter steerable functions using cascade basis reduction , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[2]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[4]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[5]  F. Natterer.  The Mathematics of Computerized Tomography , 1986 .

[6]  Tim Weyrich,et al.  A system for high-volume acquisition and matching of fresco fragments: reassembling Theran wall paintings , 2008, ACM Trans. Graph..

[7]  Steven G. Johnson,et al.  The Design and Implementation of FFTW3 , 2005, Proceedings of the IEEE.

[8]  J. García-cuerva,et al.  Fourier Analysis and Partial Differential Equations , 2001 .

[9]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[10]  Ben Weiss.  Fast median and bilateral filtering , 2006, SIGGRAPH 2006.

[11]  Jont B. Allen,et al.  Short term spectral analysis, synthesis, and modification by discrete Fourier transform , 1977 .

[12]  Jos Starn.  A Simple Fluid Solver Based on the FFT , 2001, J. Graphics, GPU, & Game Tools.

[13]  Michael Potmesil,et al.  A lens and aperture camera model for synthetic image generation , 1981, SIGGRAPH '81.

[14]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[15]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Jiri Matas,et al.  Working hard to know your neighbor's margins: Local descriptor learning loss , 2017, NIPS.

[17]  Benjamin B. Kimia,et al.  Archaeological Fragment Reconstruction Using Curve-Matching , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  Lei Zhou,et al.  GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints , 2018, ECCV.

[19]  Gregory S. Chirikjian,et al.  Pattern Matching as a Correlation on the Discrete Motion Group , 1999, Comput. Vis. Image Underst..

[20]  Joachim Weickert,et al.  A Review of Nonlinear Diffusion Filtering , 1997, Scale-Space.

[21]  Torsten Sattler,et al.  Comparative Evaluation of Hand-Crafted and Learned Local Features , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Gavin S. P. Miller,et al.  Rapid, stable fluid dynamics for computer graphics , 1990, SIGGRAPH.

[23]  Shaharyar Ahmed Khan Tareen,et al.  A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK , 2018, 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET).

[24]  Helmut Pottmann,et al.  Reassembling fractured objects by geometric matching , 2006, ACM Trans. Graph..

[25]  Szymon Rusinkiewicz,et al.  Learning Local Descriptors With a CDF-Based Dynamic Soft Margin , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[28]  Eero P. Simoncelli,et al.  Steerable wedge filters for local orientation analysis , 1996, IEEE Trans. Image Process..

[29]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[30]  Yoel Shkolnisky,et al.  A signal processing approach to symmetry detection , 2006, IEEE Transactions on Image Processing.

[31]  Pascal Fua,et al.  Image Matching Across Wide Baselines: From Paper to Practice , 2020, International Journal of Computer Vision.

[32]  J.A. Moorer,et al.  Signal processing aspects of computer music: A survey , 1977, Proceedings of the IEEE.

[33]  Adam Finkelstein,et al.  High-Precision Localization Using Ground Texture , 2017, 2019 International Conference on Robotics and Automation (ICRA).

[34]  Yan Lu,et al.  Local Descriptors Optimized for Average Precision , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Hans-Peter Seidel,et al.  Adaptive Fourier-Based Surface Reconstruction , 2006, GMP.

[36]  Randall R. Holmes.  Linear Representations of Finite Groups , 2008 .

[37]  B. V. K. Vijaya Kumar,et al.  Correlation Pattern Recognition , 2002 .

[38]  M. Glas,et al.  Principles of Computerized Tomographic Imaging , 2000 .

[39]  Mohamed S. Shehata,et al.  Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images , 2017, ArXiv.

[40]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[41]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[42]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[43]  Szymon Rusinkiewicz,et al.  Symmetry descriptors and 3D shape matching , 2004, SGP '04.

[44]  Jorge Stolfi,et al.  A Multiscale Method for the Reassembly of Two-Dimensional Fragmented Objects , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.

[46]  John D. Owens,et al.  Interactive Depth of Field Using Simulated Diffusion on a GPU , 2006 .

[47]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[48]  R. Wilson,et al.  Anisotropic Nonstationary Image Estimation and Its Applications: Part I - Restoration of Noisy Images , 1983, IEEE Transactions on Communications.