The Fractional Relative Capacity and the Fractional Laplacian with Neumann and Robin Boundary Conditions on Open Sets

Let Ω⊂ℝN$\Omega \subset \mathbb {R}^{N}$ be an arbitrary open set with boundary ∂Ω. Let p∈[1,∞)$p\in [1,\infty )$ and s∈(0,1). In the first part of the article we give some useful properties of the fractional order Sobolev spaces. We define a relative (s,p)-capacity on Ω¯$\overline {\Omega }$ with the fractional order Sobolev spaces, give its properties and its connection with the classical Bessel (s,p)-capacity and the Hausdorff measure. We also use the relative capacity to characterize completely the zero trace fractional order Sobolev spaces. In the second part of the article, we clarify the Neumann and Robin boundary conditions associated with the fractional Laplace operator on open subsets of ℝN$\mathbb {R}^{N}$. Contrary to the classical Laplace operator, it turns out that Dirichlet, Neumann and Robin boundary conditions may coincide for the fractional Laplacian on bounded domains. In the last part of the article we consider some nonlocal elliptic problems associated with the fractional Laplacian with Neumann and Robin type boundary conditions. We show some existence and regularity results of weak solutions on non smooth domains.