A blob method method for inhomogeneous diffusion with applications to multi-agent control and sampling