Realization of Stochastic Systems

With each state of probabilistic finite automaton, we may associate a function p(u|v), which tells us the probability that, if we apply the input sequence v to the machine started in the state, we should observe output sequence u. We give a necessary and sufficient condition that a function p(u|v) be realizable as such an inputoutput function. We show that a result of Heller on stochastic processes derived from Markov chains is a special case of our condition.