Thermal fading of the $1/k^4$-tail of the momentum distribution induced by the hole anomaly