In this paper we discuss the recovery of derivatives and the computation of rigorous and useful upper bounds for the pointwise error in the recovered derivatives, for finite element approximations of the Laplace equation with Neumann boundary conditions, especially at points close to or on a smooth, curved boundary. We analyze the dipole image technique for the case of curved boundaries, and show how to compute reliable recovered derivatives and error bounds even in the limiting case of points lying on the curved boundary. Numerical experiments show reasonably tight error bounds for points both close to and away from a curved boundary.