The effect of line damping, magneto-optics and parasitic light on the derivation of sunspot vector magnetic fields