Quick Start : Resolving a Markov decision process problem using the MDPtoolbox in Matlab

The probability that the process moves into its new state s’ is influenced by the chosen action. Specifically, it is given by the state transition function P (s, s′, a). Thus, the next state s’ depends on the current state s and the decision maker’s action a. But given s and a, it is conditionally independent of all previous states and actions; in other words, the state transitions of an MDP possess the Markov property.