Optimal Prediction of Markov Chains With and Without Spectral Gap