Instabilities of Offline RL with Pre-Trained Neural Representation