An Instance-Based State Representation for Network Repair

We describe a formal framework for diagnosis and repair problems that shares elements of the well known partially observable MOP and cost-sensitive classification models. Our cost-sensitive fault remediation model is amenable to implementation as a reinforcement-learning system, and we describe an instance-based state representation that is compatible with learning and planning in this framework. We demonstrate a system that uses these ideas to learn to efficiently restore network connectivity after a failure.

[1]  Christopher G. Atkeson,et al.  Memory-Based Learning Control , 1991, 1991 American Control Conference.

[2]  Lonnie Chrisman,et al.  Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach , 1992, AAAI.

[3]  Avrim Blum,et al.  New approximation algorithms for graph coloring , 1994, JACM.

[4]  Peter D. Turney.  Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm , 1994, J. Artif. Intell. Res..

[5]  Andrew McCallum,et al.  Instance-Based Utile Distinctions for Reinforcement Learning with Hidden State , 1995, ICML.

[6]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[7]  Dan Roth,et al.  Learning Active Classifiers , 1996, ICML.

[8]  Ashwin Ram,et al.  Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces , 1997, Adapt. Behav..

[9]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[10]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[11]  Leslie Pack Kaelbling,et al.  Practical Reinforcement Learning in Continuous Spaces , 2000, ICML.

[12]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[13]  Richard P. Martin,et al.  Mendosis: A SAN-based Fault Injection Test-bed for the Construction of Highly Available Network Services , 2001 .

[14]  Thomas G. Dietterich,et al.  Pruning Improves Heuristic Search for Cost-Sensitive Learning , 2002, ICML.

[15]  AnYuan Guo.  Active Classification with Bounded Resources , 2002 .

[16]  Yishay Mansour,et al.  A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes , 1999, Machine Learning.

[17]  Liviu Iftode,et al.  Remote Repair of OS State Using Backdoors , 2004 .

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  Liming Xiang,et al.  Kernel-Based Reinforcement Learning , 2006, ICIC.