Diffusion of Cosmic Rays in the Expanding Universe. II. Energy Spectra of Ultra-High Energy Cosmic Rays