Gelfand numbers related to structured sparsity and Besov space embeddings with small mixed smoothness

Abstract We consider the problem of determining the asymptotic order of the Gelfand numbers of mixed-(quasi-)norm embeddings l p b ( l q d ) ↪ l r b ( l u d ) given that p ≤ r and q ≤ u , with emphasis on cases with p ≤ 1 and/or q ≤ 1 . These cases turn out to be related to structured sparsity. We obtain sharp bounds in a number of interesting parameter constellations. Our new matching bounds for the Gelfand numbers of the embeddings of l 1 b ( l 2 d ) and l 2 b ( l 1 d ) into l 2 b ( l 2 d ) imply optimality assertions for the recovery of block-sparse and sparse-in-levels vectors, respectively. In addition, we apply our sharp estimates for l p b ( l q d ) -spaces to obtain new two-sided estimates for the Gelfand numbers of multivariate Besov space embeddings in regimes of small mixed smoothness. It turns out that in some particular cases these estimates show the same asymptotic behavior as in the univariate situation. In the remaining cases they differ at most by a log log factor from the univariate bound.

[1]  Van Kien Nguyen,et al.  Gelfand numbers of embeddings of mixed Besov spaces , 2016, J. Complex..

[2]  W. A. Light.  n -WIDTHS IN APPROXIMATION THEORY (Ergebnisse der Mathematik und ihrer Grenzgebiete 3. Folge, Band 7) , 1985 .

[3]  T. Ullrich,et al.  Optimal sampling recovery of mixed order Sobolev embeddings via discrete Littlewood–Paley type characterizations , 2016, 1603.04809.

[4]  A. D. Izaak.  Widths of Hölder-Nikol'skii classes and finite-dimensional subsets in spaces with mixed norm , 1996 .

[5]  S. B. Stechkin.  On absolute convergence of orthogonal series. I. , 1953 .

[6]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[7]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[8]  V. N. Temlyakov,et al.  Constructive Sparse Trigonometric Approximation for Functions with Small Mixed Smoothness , 2015, 1503.00282.

[9]  I. Daubechies,et al.  Capturing Ridge Functions in High Dimensions from Point Queries , 2012 .

[10]  A. Pietsch.  Eigenvalues and S-Numbers , 1987 .

[11]  E. Gilbert.  A comparison of signalling alphabets , 1952 .

[12]  Mario Ullrich,et al.  The Role of Frolov's Cubature Formula for Functions with Bounded Mixed Derivative , 2015, SIAM J. Numer. Anal..

[13]  Thomas Kühn,et al.  A Lower Estimate for Entropy Numbers , 2001, J. Approx. Theory.

[14]  A. Pinkus.  n-Widths in Approximation Theory , 1985 .

[15]  Martin J. Wainwright,et al.  A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers , 2009, NIPS.

[16]  Y. Gordon.  On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[17]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[18]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[19]  Yonina C. Eldar,et al.  Block-sparsity: Coherence and efficient recovery , 2008, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[21]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[22]  É. M. Galeev.  Linear widths of Hölder-Nikol'skii classes of periodic functions of several variables , 1996 .

[23]  Hans Triebel,et al.  Function Spaces with Dominating Mixed Smoothness , 2019 .

[24]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[25]  B. S. Kašin,et al.  DIAMETERS OF SOME FINITE-DIMENSIONAL SETS AND CLASSES OF SMOOTH FUNCTIONS , 1977 .

[26]  Anders C. Hansen,et al.  On the absence of the RIP in real-world applications of compressed sensing and the RIP in levels , 2014, ArXiv.

[27]  H. Rauhut,et al.  Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms , 2008 .

[28]  Winfried Sickel,et al.  Weyl numbers of embeddings of tensor product Besov spaces , 2014, J. Approx. Theory.

[29]  Junzhou Huang,et al.  The Benefit of Group Sparsity , 2009 .

[30]  V. N. Temli︠a︡kov.  Approximation of functions with bounded mixed derivative , 1989 .

[31]  S. Geer,et al.  Oracle Inequalities and Optimal Inference under Group Sparsity , 2010, 1007.1771.

[32]  Sebastian Mayer,et al.  Counting Via Entropy: New Preasymptotics for the Approximation Numbers of Sobolev Embeddings , 2015, SIAM J. Numer. Anal..

[33]  Holger Rauhut,et al.  The Gelfand widths of ℓp-balls for 0 , 2010, ArXiv.

[34]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[35]  D. L. Donoho,et al.  Compressed sensing , 2006, IEEE Trans. Inf. Theory.

[36]  Winfried Sickel,et al.  Best m-Term Approximation and Sobolev–Besov Spaces of Dominating Mixed Smoothness—the Case of Compact Embeddings , 2012 .

[37]  A. D. Izaak.  Kolmogorov widths in finite-dimensional spaces with mixed norms , 1994 .

[38]  Holger Rauhut,et al.  The Gelfand widths of lp-balls for 0p<=1 , 2010, J. Complex..

[39]  Jan Vybíral,et al.  Entropy and Sampling Numbers of Classes of Ridge Functions , 2013, 1311.2005.

[40]  Andrej Yu. Garnaev,et al.  On widths of the Euclidean Ball , 1984 .

[41]  The product of octahedra is badly approximated in the ℓ2,1-metric , 2017 .

[42]  David E. Edmunds,et al.  Gelfand numbers and widths , 2013, J. Approx. Theory.

[43]  Tosio Aoki.  117. Locally Bounded Linear Topological Spaces , 1942 .

[44]  Jan Vybíral,et al.  Learning Functions of Few Arbitrary Linear Parameters in High Dimensions , 2010, Found. Comput. Math..

[45]  Vladimir N. Temlyakov,et al.  Hyperbolic Cross Approximation , 2016, 1601.03978.

[46]  Jan Vybíral.  Function spaces with dominating mixed smoothness , 2006 .

[47]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[48]  Ben Adcock,et al.  Breaking the coherence barrier: asymptotic incoherence and asymptotic sparsity in compressed sensing , 2013, ArXiv.

[49]  H. Triebel,et al.  Topics in Fourier Analysis and Function Spaces , 1987 .

[50]  A. A. Vasil'eva,et al.  Kolmogorov and linear widths of the weighted Besov classes with singularity at the origin , 2012, J. Approx. Theory.

[51]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[52]  Chen Li,et al.  Compressed sensing with local structure: uniform recovery guarantees for the sparsity in levels class , 2016, Applied and Computational Harmonic Analysis.

[53]  H. Woxniakowski.  Information-Based Complexity , 1988 .

[54]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[55]  É. M. Galeev,et al.  KOLMOGOROV WIDTHS OF CLASSES OF PERIODIC FUNCTIONS OF ONE AND SEVERAL VARIABLES , 1991 .

[56]  Aicke Hinrichs,et al.  Carl's inequality for quasi-Banach spaces , 2015, 1512.04421.

[57]  Yaniv Plan,et al.  One‐Bit Compressed Sensing by Linear Programming , 2011, ArXiv.

[58]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .

[59]  Jan Vybíral,et al.  Widths of embeddings in function spaces , 2008, J. Complex..

[60]  Holger Rauhut,et al.  Uniform recovery of fusion frame structured sparse signals , 2014, ArXiv.

[61]  Pierre Weiss,et al.  Compressed sensing with structured sparsity and structured acquisition , 2015, Applied and Computational Harmonic Analysis.

[62]  Product of octahedra is badly approximated in the $\ell_{2,1}$-metric , 2016, 1606.00738.

[63]  M. Wainwright.  Structured Regularizers for High-Dimensional Problems: Statistical and Computational Issues , 2014 .

[64]  Yaniv Plan,et al.  Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach , 2012, IEEE Transactions on Information Theory.

[65]  Ben Adcock,et al.  BREAKING THE COHERENCE BARRIER: A NEW THEORY FOR COMPRESSED SENSING , 2013, Forum of Mathematics, Sigma.