Random Markov processes for countable and uncountable alphabets