Proof of proposition 1

Before we look at the MMD calculations in various cases, we prove the following useful characterization of MMD for translation invariant kernels like the Gaussian and Laplace kernels. Lemma 1. For translation invariant kernels, there exists a pdf s such that MMD 2 (p, q) = s(w)|Φ p (w) − Φ q (w)| 2 dw, where Φ p , Φ q denote the characteristic functions of p, q respectively. Proof. From definition of MMD 2 , we have MMD 2 (p, q) = x,x k(x, x)p(x)p(x)dxdx + x,x k(x, x)q(x)q(x)dxdx − 2 x,x k(x, x)p(x)q(x)dxdx. From Bochner's theorem (see (Rudin 1962)) for translation invariant kernels, we know k(x, x) = w s(w)e iw x e −iw x dw where s is the fourier transform of the kernel. Substituting the above equality in the definition of MMD 2 , we have the required result. Proof. Since Gaussian kernel is a translation invariant kernel, we can use Lemma 1 to derive the MMD 2 in this case. It is well-known that the Fourier transform s(w) of Gaussian kernel is Gaussian distribution. Substituting the characteristic function of normal distribution in Lemma 1, we have MMD 2 (p, q) = w γ 2 /2π d/2 exp −γ 2 w 2 /2 exp(iµ 1 w − w Σw/2) − exp(iµ 1 w − w Σw/2) 2 dw = γ 2 /2π d/2 w exp −w Σw exp −γ 2 w 2 /2 exp(iµ 1 w) − exp(iµ 2 w) 2 dw = γ 2 /2π d/2 w exp −w (Σ + γ 2 I/2)w 2 − exp −i(µ 1 − µ 2) w − exp −i(µ 2 − µ 1) w dw = 2 γ 2 /2π d/2 w exp −w (Σ + γ 2 I/2)w 1 − exp −i(µ 1 − µ 2) w dw (2) The third step follows from definition of complex conjugate. In what follows, we do the following change of variable u = (Σ + γ 2 I/2)