Markov decision processes with their applications

Discretetimemarkovdecisionprocesses: Total Reward.- Discretetimemarkovdecisionprocesses: Average Criterion.- Continuous Time Markov Decision Processes.- Semi-Markov Decision Processes.- Markovdecisionprocessesinsemi-Markov Environments.- Optimal control of discrete event systems: I.- Optimal control of discrete event systems: II.- Optimal replacement under stochastic Environments.- Optimalal location in sequential online Auctions.