Decoding the hologram: Scalar fields interacting with gravity