A Torres condition for twisted Alexander polynomials