Were small galaxies once the dominant cosmological population?