The inverse diffusion time scale of velocity gradients in homogeneous isotropic turbulence