The cosmological evolution of the average mass per baryon