Reproducing Kernel Hilbert Spaces Approximation Bounds

We find probability error bounds for approximations of functions $f$ in a separable reproducing kernel Hilbert space $\mathcal{H}$ with reproducing kernel $K$ on a base space $X$, firstly in terms of finite linear combinations of functions of type $K_{x_i}$ and then in terms of the projection $\pi^n_x$ on $\mathrm{Span}\{K_{x_i} \}^n_{i=1}$, for random sequences of points $x=(x_i)_i$ in the base space $X$. Previous results demonstrate that, for sequences of points $(x_i)_{i=1}^\infty$ constituting a so-called \emph{uniqueness set}, the orthogonal projections $\pi^n_x$ to $\mathrm{Span}\{K_{x_i} \}^n_{i=1}$ converge in the strong operator topology to the identity operator. The main result shows that, for a given probability measure $P$, letting $P_K$ be the measure defined by $\mathrm{d} P_K(x)=K(x,x)\mathrm{d} P(x)$, $x\in X$, and $\mathrm{H}_P$ denote the reproducing kernel Hilbert space that is the operator range of the nonexpansive operator \begin{equation*} L^2(X;P_K)\ni \lambda\mapsto L_{P,K}\lambda:=\int_X \lambda(x)K_x\mathrm{d} P(x)\in \mathcal{H}, \end{equation*} where the integral exists in the Bochner sense, under the assumption that $\mathcal{H}_P$ is dense in $\mathcal{H}$ any sequence of points sampled independently from $P$ yields a uniqueness set with probability $1$. This result improves on previous error bounds in weaker norms, such as uniform or $L^p$ norms, which yield only convergence in probability and not almost certain convergence. Two examples that show the applicability of this result to a uniform distribution on a compact interval and to the Hardy space $H^2(\mathbb{D})$ are presented as well.