Eigenvalue conjecture and colored Alexander polynomials