Holographic Relative Entropy in Infinite-Dimensional Hilbert Spaces