Using the Manhattan distance for computing the multiobjective Markov chains problem

ABSTRACT This paper presents a novel method for computing the multi-objective problem in the case of a metric state space using the Manhattan distance. The problem is restricted to a class of ergodic controllable finite Markov chains. This optimization approach is developed for converging to an optimal solution that corresponds to a strong Pareto optimal point in the Pareto front. The method consists of a two-step iterated procedure: (a) the first step consists on an approximation to a strong Pareto optimal point and, (b) the second step is a refinement of the previous approximation. We formulate the problem adding the Tikhonov's regularization method to ensure the convergence of the cost-functions to a unique strong point into the Pareto front. We prove that there exists an optimal solution that is a strong Pareto optimal solution and it is the closest solution to the utopian point of the Pareto front. The proposed solution is validated theoretically and by a numerical example considering the vehicle routing planning problem.

[1]  Werner C. Rheinboldt,et al.  A Program for a Locally-Parametrized Continuation Process. , 1981 .

[2]  Werner C. Rheinboldt,et al.  Algorithm 596: a program for a locally parameterized , 1983, TOMS.

[3]  Michael N. Katehakis,et al.  Multiobjective Markov decisions in urban modelling , 1985 .

[4]  K. Tanaka,et al.  The closest solution to the shadow minimum of a cooperative dynamic game , 1989 .

[5]  R. T. Haftka,et al.  Tracing the Efficient Curve for Multi-objective Control-Structure Optimization , 1991 .

[6]  Layne T. Watson,et al.  Multi-Objective Control-Structure Optimization via Homotopy Methods , 1993, SIAM J. Optim..

[7]  A. Tikhonov,et al.  Numerical Methods for the Solution of Ill-Posed Problems , 1995 .

[8]  J. Dennis,et al.  NORMAL-BOUNDARY INTERSECTION: AN ALTERNATE METHOD FOR GENERATING PARETO OPTIMAL POINTS IN MULTICRITERIA OPTIMIZATION PROBLEMS , 1996 .

[9]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[10]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[11]  K. Wakuta,et al.  Solution procedures for multi-objective markov decision processes , 1998 .

[12]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[13]  Xin Yao,et al.  Parallel Problem Solving from Nature PPSN VI , 2000, Lecture Notes in Computer Science.

[14]  Alexander S. Poznyak,et al.  Self-Learning Control of Finite Markov Chains , 2000 .

[15]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[16]  Jonathan E. Fieldsend,et al.  A Multi-Objective Algorithm based upon Particle Swarm Optimisation, an Efficient Data Structure and , 2002 .

[17]  Sanaz Mostaghim.  Multi-objective evolutionary algorithms: data structures, convergence, and diversity , 2004 .

[18]  M. Dellnitz,et al.  Covering Pareto Sets by Multilevel Subdivision Techniques , 2005 .

[19]  Srini Narayanan,et al.  Learning all optimal policies with multiple criteria , 2008, ICML '08.

[20]  Hisao Ishibuchi,et al.  Evolutionary many-objective optimization: A short review , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[21]  Lothar Thiele,et al.  Quality Assessment of Pareto Set Approximations , 2008, Multiobjective Optimization.

[22]  Andrei V. Kelarev,et al.  Constructing Stochastic Mixture Policies for Episodic Multiobjective Reinforcement Learning Tasks , 2009, Australasian Conference on Artificial Intelligence.

[23]  Víctor Pereyra,et al.  Fast computation of equispaced Pareto manifolds and Pareto fronts for multiobjective optimization problems , 2009, Math. Comput. Simul..

[24]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[25]  S. Utyuzhnikov,et al.  Directed search domain: a method for even generation of the Pareto frontier in multiobjective optimization , 2011 .

[26]  Rajeev Kumar,et al.  Characterization of graph properties for improved Pareto fronts using heuristics and EA for bi-objective graph coloring problem , 2013, Appl. Soft Comput..

[27]  Shimon Whiteson,et al.  A Survey of Multi-Objective Sequential Decision-Making , 2013, J. Artif. Intell. Res..

[28]  José Castillo,et al.  Equispaced Pareto front construction for constrained bi-objective optimization , 2013, Math. Comput. Model..

[29]  Alexander S. Poznyak,et al.  Simple computing of the customer lifetime value: A fixed local-optimal policy approach , 2014, Journal of Systems Science and Systems Engineering.

[30]  Peter J. Fleming,et al.  Pareto Front Estimation for Decision Making , 2014, Evolutionary Computation.

[31]  Shimon Whiteson,et al.  Bounded Approximations for Linear Multi-Objective Planning Under Uncertainty , 2014, ICAPS.

[32]  Kalyanmoy Deb,et al.  Multi-Objective Evolutionary Algorithms , 2015, Handbook of Computational Intelligence.

[33]  Marcello Restelli,et al.  Multi-Objective Reinforcement Learning with Continuous Pareto Frontier Approximation , 2014, AAAI.

[34]  Alexander S. Poznyak,et al.  Computing the Stackelberg/Nash equilibria using the extraproximal method: Convergence analysis and implementation details for Markov chains games , 2015, Int. J. Appl. Math. Comput. Sci..

[35]  Julio B. Clempner.  Necessary and sufficient Karush-Kuhn-Tucker conditions for multiobjective Markov chains optimality , 2016, Autom..

[36]  Alexander S. Poznyak,et al.  Solving the Pareto front for multiobjective Markov chains using the minimum Euclidean distance gradient-based optimization method , 2016, Math. Comput. Simul..