Observing 4d baby universes in quantum gravity