Explicit Representations for Banach Subspaces of Lizorkin Distributions

The Lizorkin space is well-suited for studying various operators; e.g., fractional Laplacians and the Radon transform. In this paper, we show that the space is unfortunately not complemented in the Schwartz space. However, we can show that it is dense in $C_0(\mathbb R^d)$, a property that is shared by the larger Schwartz space and that turns out to be useful for applications. Based on this result, we investigate subspaces of Lizorkin distributions that are Banach spaces and for which a continuous representation operator exists. Then, we introduce a variational framework involving these spaces and that makes use of the constructed operator. By investigating two particular cases of this framework, we are able to strengthen existing results for fractional splines and 2-layer ReLU networks.

[1]  L. Rosasco,et al.  Understanding neural networks with reproducing kernel Banach spaces , 2021, Applied and Computational Harmonic Analysis.

[2]  M. Burger,et al.  A Bregman Learning Framework for Sparse Neural Networks , 2021, J. Mach. Learn. Res..

[3]  Robert D. Nowak,et al.  What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory , 2021, SIAM J. Math. Data Sci..

[4]  Michael Unser,et al.  Convex Optimization in Sums of Banach Spaces , 2021, 2104.13127.

[5]  Robert D. Nowak,et al.  Banach Space Representer Theorems for Neural Networks and Ridge Splines , 2020, J. Mach. Learn. Res..

[6]  Matthieu Simeoni,et al.  TV-based reconstruction of periodic functions , 2020, Inverse Problems.

[7]  N. Teofanov,et al.  The Shearlet Transform and Lizorkin Spaces , 2020, Landscapes of Time-Frequency Analysis.

[8]  Nathan Srebro,et al.  A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case , 2019, ICLR.

[9]  Pierre Weiss,et al.  On the linear convergence rates of exchange and continuous methods for total variation minimization , 2019, Mathematical Programming.

[10]  Michael Unser,et al.  B-Spline-Based Exact Discretization of Continuous-Domain Inverse Problems With Generalized TV Regularization , 2019, IEEE Transactions on Information Theory.

[11]  Michael Unser,et al.  A Unifying Representer Theorem for Inverse Problems and Machine Learning , 2019, Foundations of Computational Mathematics.

[12]  Nathan Srebro,et al.  How do infinite width bounded norm networks look in function space? , 2019, COLT.

[13]  Jun Zhang,et al.  On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructions , 2019, Acta Mathematica Sinica, English Series.

[14]  Emmanuel Soubies,et al.  The sliding Frank–Wolfe algorithm and its application to super-resolution microscopy , 2018, Inverse Problems.

[15]  K. Bredies,et al.  Sparsity of solutions for variational inverse problems with finite-dimensional data , 2018, Calculus of Variations and Partial Differential Equations.

[16]  P. R. Stinga,et al.  User’s guide to the fractional Laplacian and the method of semigroups , 2018, Fractional Differential Equations.

[17]  Michael Unser,et al.  Continuous-Domain Solutions of Linear Inverse Problems With Tikhonov Versus Generalized TV Regularization , 2018, IEEE Transactions on Signal Processing.

[18]  Heinrich Müller,et al.  SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  J. P. Ward,et al.  Beyond Wiener’s Lemma: Nuclear Convolution Algebras and the Inversion of Digital Filters , 2017, Journal of Fourier Analysis and Applications.

[20]  Michael Unser,et al.  Splines Are Universal Solutions of Linear Inverse Problems with Generalized TV Regularization , 2016, SIAM Rev..

[21]  P. L. Combettes,et al.  Regularized Learning Schemes in Feature Banach Spaces , 2014, 1410.6847.

[22]  G. Shilov,et al.  Properties and Operations , 2014 .

[23]  Stevan Pilipovic,et al.  The Ridgelet transform of distributions , 2013, 1306.2024.

[24]  Donald Ludwig,et al.  The radon transform on euclidean space , 2010 .

[25]  Wen Yuan,et al.  Morrey and Campanato Meet Besov, Lizorkin and Triebel , 2010, Lecture Notes in Mathematics.

[26]  Yuesheng Xu,et al.  Reproducing kernel Banach spaces for machine learning , 2009, 2009 International Joint Conference on Neural Networks.

[27]  곽순섭,et al.  Generalized Functions , 2006, Theoretical and Mathematical Physics.

[28]  S. Samko.  Hypersingular Integrals and Their Applications , 2001 .

[29]  Thierry Blu,et al.  Fractional Splines and Wavelets , 2000, SIAM Rev..

[30]  Boris Rubin,et al.  Fractional Integrals and Potentials , 1996 .

[31]  O. Marichev,et al.  Fractional Integrals and Derivatives: Theory and Applications , 1993 .

[32]  F. Trèves.  Topological vector spaces, distributions and kernels , 1967 .

[33]  A. Grothendieck,et al.  Produits Tensoriels Topologiques Et Espaces Nucleaires , 1966 .

[34]  Sigurdur Helgason,et al.  The Radon transform on Euclidean spaces, compact two-point homogeneous spaces and Grassmann manifolds , 1965 .

[35]  Michael Unser,et al.  Learning Activation Functions in Deep (Spline) Neural Networks , 2020, IEEE Open Journal of Signal Processing.

[36]  Kristian Kirsch,et al.  Methods Of Modern Mathematical Physics , 2016 .

[37]  Paul B. Garrett.  Topological vector spaces , 2016 .

[38]  C. Stolk.  The Radon transform , 2014 .

[39]  Stefan Samko,et al.  Denseness of the spaces $Φ_V$ of Lizorkin type in the mixed $L^{p̅}(ℝ^n)$-spaces , 1995 .

[40]  I. J. Schoenberg.  Contributions to the Problem of Approximation of Equidistant Data by Analytic Functions , 1988 .

[41]  L. Schwartz.  Théorie des distributions , 1966 .