Residual properties of 3-manifold groups I: Fibered and hyperbolic 3-manifolds