Random Subsets Support Learning a Mixture of Heuristics

Problem solvers, both human and machine, have at their disposal many heuristics that may support effective search. The efficacy of these heuristics, however, varies with the problem class, and their mutual interactions may not be well understood. The long-term goal of our work is to learn how to select appropriately from among a large body of heuristics, and how to combine them into a mixture that works well on a specific class of problems. The principal result reported here is that randomly chosen subsets of heuristics can improve the identification of an appropriate mixture of heuristics. A self-supervised learner uses this method here to learn to solve constraint satisfaction problems quickly and effectively.

[1]  Lars Otten,et al.  Randomization in Constraint Programming for Airline Planning , 2006, CP.

[2]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[3]  J. Mesirov,et al.  Hybrid system for protein secondary structure prediction. , 1992, Journal of molecular biology.

[4]  Richard J. Wallace Analysis of Heuristic Synergies , 2005, CSCLP.

[5]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[6]  Ethem Alpaydin,et al.  Combining multiple representations and classifiers for pen-based handwritten digit recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[7]  Eugene C. Freuder,et al.  Understanding and Improving the MAC Algorithm , 1997, CP.

[8]  Carla P. Gomes,et al.  Randomized Backtrack Search , 2004 .

[9]  Pierre Flener,et al.  Towards Inferring Labelling Heuristics for CSP Application Domains , 2001, KI/ÖGAI.

[10]  Toby Walsh,et al.  The Constrainedness of Search , 1996, AAAI/IAAI, Vol. 1.

[11]  Inês Lynce,et al.  Stochastic Systematic Search Algorithms for Satisfiability , 2001, Electron. Notes Discret. Math..

[12]  Bart Selman,et al.  Local search strategies for satisfiability testing , 1993, Cliques, Coloring, and Satisfiability.

[13]  Smiljana Petrovic,et al.  Relative Support Weight Learning for Constraint Solving , 2006 .

[14]  Barbara M. Smith,et al.  Trying Harder to Fail First , 1998, ECAI.

[15]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[16]  Barry O'Sullivan,et al.  Search Heuristics and Heavy-Tailed Behaviour , 2005, CP.

[17]  John A. Allen,et al.  An Overview of Learning in the Multi-TAC System , 2007 .

[18]  Susan L. Epstein,et al.  Full Restart Speeds Learning , 2006, FLAIRS Conference.

[19]  Carlo Mannino,et al.  Models and solution techniques for frequency assignment problems , 2003, 4OR.

[20]  M. Pazzani,et al.  Error Reduction through Learning Multiple Descriptions , 1996, Machine Learning.

[21]  Christophe Lecoutre,et al.  Backjump-based techniques versus conflict-directed heuristics , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[22]  Barbara M. Smith,et al.  Symmetry Breaking in Graceful Graphs , 2003, CP.

[23]  Susan L. Epstein For the Right Reasons: The FORR Architecture for Learning in a Skill Domain , 1994, Cogn. Sci..

[24]  Rina Dechter,et al.  Look-Ahead Value Ordering for Constraint Satisfaction Problems , 1995, IJCAI.

[25]  Lakhdar Sais,et al.  Boosting Systematic Search by Weighting Constraints , 2004, ECAI.

[26]  Susan L. Epstein,et al.  LEARNING TO SUPPORT CONSTRAINT PROGRAMMERS , 2005, Comput. Intell..

[27]  Eugene C. Freuder,et al.  Contradicting Conventional Wisdom in Constraint Satisfaction , 1994, ECAI.

[28]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Bart Selman,et al.  Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems , 2000, Journal of Automated Reasoning.

[30]  Susan L. Epstein Metaknowledge for Autonomous Systems , 2004 .

[31]  Christian Bessiere,et al.  Statistical Regimes Across Constrainedness Regions , 2004, Constraints.

[32]  Giorgio Valentini,et al.  Ensembles of Learning Machines , 2002, WIRN.

[33]  David W. Opitz,et al.  Generating Accurate and Diverse Members of a Neural-Network Ensemble , 1995, NIPS.

[34]  Manfred K. Warmuth,et al.  Averaging Expert Predictions , 1999, EuroCOLT.

[35]  Christian Bessiere,et al.  MAC and Combined Heuristics: Two Reasons to Forsake FC (and CBJ?) on Hard Problems , 1996, CP.