Inferring Rewards from Language in Context