The Evolving Interstellar Medium of Star-forming Galaxies, as Traced by Stardust