A Tutorial on the Cross-Entropy Method

The cross-entropy (CE) method is a new generic approach to combinatorial and multi-extremal optimization and rare event simulation. The purpose of this tutorial is to give a gentle introduction to the CE method. We present the CE methodology, the basic algorithm and its modifications, and discuss applications in combinatorial optimization and machine learning.

[1]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[2]  M. Garey.  Johnson: computers and intractability: a guide to the theory of np- completeness (freeman , 1979 .

[3]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[5]  Mihalis Yannakakis,et al.  Optimization, approximation, and complexity classes , 1991, STOC '88.

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  Charles Leake,et al.  Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method , 1994 .

[8]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[9]  Giovanni Righini,et al.  Heuristics from Nature for Hard Combinatorial Optimization Problems , 1996 .

[10]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[11]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[12]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[13]  Reuven Y. Rubinstein,et al.  Optimization of computer simulation models with rare events , 1997 .

[14]  Reuven Y. Rubinstein,et al.  Modern simulation and modeling , 1998 .

[15]  C. Voudouris,et al.  Guided Local Search — an Illustrative Example in Function Optimisation , 1998 .

[16]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.

[17]  R. Rubinstein.  The Cross-Entropy Method for Combinatorial and Continuous Optimization , 1999 .

[18]  John N. Tsitsiklis,et al.  Actor-Critic Algorithms , 1999, NIPS.

[19]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[20]  Walter J. Gutjahr,et al.  A Graph-based Ant System and its convergence , 2000, Future Gener. Comput. Syst..

[21]  Pieter Tjerk de Boer,et al.  Analysis and efficient simulation of queueing models of telecommunications systems , 2000 .

[22]  Leyuan Shi,et al.  Nested Partitions Method for Global Optimization , 2000, Oper. Res..

[23]  Andrew G. Barto,et al.  Robot Weightlifting By Direct Policy Search , 2001, IJCAI.

[24]  Bjarne E. Helvik,et al.  Using the Cross-Entropy Method to Guide/Govern Mobile Agent's Path Finding in Networks , 2001, MATA.

[25]  R. Rubinstein.  Combinatorial Optimization, Cross-Entropy, Ants and Rare Events , 2001 .

[26]  Peter L. Bartlett,et al.  Experiments with Infinite-Horizon, Policy-Gradient Estimation , 2001, J. Artif. Intell. Res..

[27]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[28]  Pieter-Tjerk de Boer,et al.  Estimating buffer overflows in three stages using cross-entropy , 2002, Proceedings of the Winter Simulation Conference.

[29]  Dirk P. Kroese,et al.  Sequence alignment by rare event simulation , 2002, Proceedings of the Winter Simulation Conference.

[30]  Dirk P. Kroese,et al.  SABRES: Sequence Alignment By Rare Event Simulation , 2002 .

[31]  Reuven Y. Rubinstein,et al.  Cross-entropy and rare events for maximal cut and partition problems , 2002, TOMC.

[32]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[33]  Ehl Emile Aarts,et al.  Simulated annealing and Boltzmann machines , 2003 .

[34]  Reuven Y. Rubinstein,et al.  Rare event estimation for static models via cross-entropy and importance sampling , 2003 .

[35]  Shie Mannor,et al.  The Cross Entropy Method for Fast Policy Search , 2003, ICML.

[36]  Vijay R. Konda,et al.  OnActor-Critic Algorithms , 2003, SIAM J. Control. Optim..

[37]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[38]  Dirk P. Kroese,et al.  The Transform Likelihood Ratio Method for Rare Event Simulation with Heavy Tails , 2004, Queueing Syst. Theory Appl..

[39]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[40]  Dirk P. Kroese,et al.  A Fast Cross-Entropy Method for Estimating Buffer Overflows in Queueing Networks , 2004, Manag. Sci..

[41]  Dirk P. Kroese,et al.  HEAVY TAILS, IMPORTANCE SAMPLING AND CROSS–ENTROPY , 2005 .

[42]  Dirk P. Kroese,et al.  Application of the Cross-Entropy Method to the Buffer Allocation Problem in a Simulation-Based Environment , 2005, Ann. Oper. Res..

[43]  Avraham Shtub,et al.  Managing Stochastic, Finite Capacity, Multi-Project Systems through the Cross-Entropy Methodology , 2005, Ann. Oper. Res..

[44]  Ad Ridder,et al.  Importance Sampling Simulations of Markovian Reliability Systems Using Cross-Entropy , 2005, Ann. Oper. Res..

[45]  Shie Mannor,et al.  Basis Function Adaptation in Temporal Difference Reinforcement Learning , 2005, Ann. Oper. Res..

[46]  L. Margolin,et al.  On the Convergence of the Cross-Entropy Method , 2005, Ann. Oper. Res..

[47]  Miro Kraetzl,et al.  The Cross-Entropy Method for Network Reliability Estimation , 2005, Ann. Oper. Res..

[48]  Tito Homem-de-Mello,et al.  Solving the Vehicle Routing Problem with Stochastic Demands using the Cross-Entropy Method , 2005, Ann. Oper. Res..

[49]  Lih-Yuan Deng,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning , 2006, Technometrics.