Roots of Ehrhart polynomials and symmetric $\delta$-vectors