When do we punish people who don’t?