Optimal Replacement Policy of Services Based on Markov Decision Process

Traditional approaches towards self-healing of services, have emphasized reactive self-healing; viz., stopping a failed service, replacing/reconfiguring it and restarting. Since this is disruptive, we investigate a proactive self-healing approach in this paper. The key issue in proactive self-healing, is the decision by the consumer, based on available observations of the performance of the provider service, to decide when to replace the currently running service (in case of atomic service)before the service fails, or when to replace a component service(in case of composite service), and in a manner that minimizes the cost involved in this replacement. In this paper we address this problem via a Markov Decision Process. We determine the optimal value and define the best action to be taken, when the consumer service is in a particular state of loss due to QoS failure of the provider service. Since the current state is known,our decision will help in proactive self-healing, by initiating service replacement earlier, and thereby minimizing losses to the consumer. We illustrate our ideas using a realistic example in the purchase order domain, and present an implementation prototype for the same.