Indications of Negative Evolution for the Sources of the Highest Energy Cosmic Rays