On the differences between plage and quiet sun in the solar photosphere