Learning-Assisted Automated Planning: Looking Back, Taking Stock, Going Forward

This article reports on an extensive survey and analysis of research work related to machine learning as it applies to automated planning over the past 30 years. Major research contributions are broadly characterized by learning method and then descriptive subcategories. Survey results reveal learning techniques that have extensively been applied and a number that have received scant attention. We extend the survey analysis to suggest promising avenues for future research in learning based on both previous experience and current needs in the planning community.

[1]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[2]  Philip J. Stone,et al.  Experiments in induction , 1966 .

[3]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[4]  Richard Fikes,et al.  Learning and Executing Generalized Robot Plans , 1993, Artif. Intell..

[5]  Earl D. Sacerdoti,et al.  Planning in a Hierarchy of Abstraction Spaces , 1974, IJCAI.

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  Laurent Siklóssy,et al.  The Role of Preprocessing in Problem Solving Systems , 1977, IJCAI.

[8]  John R. Anderson,et al.  Machine learning - an artificial intelligence approach , 1982, Symbolic computation.

[9]  Smadar T. Kedar-Cabelli,et al.  Explanation-Based Generalization as Resolution Theorem Proving , 1987 .

[10]  Jaime G. Carbonell,et al.  Learning by Experimentation , 1987 .

[11]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[12]  Oren Etzioni,et al.  Explanation-Based Learning: A Problem Solving Perspective , 1989, Artif. Intell..

[13]  Kristian J. Hammond,et al.  Case-Based Planning: Viewing Planning as a Memory Task , 1989 .

[14]  Jude Shavlik,et al.  An Approach to Combining Explanation-based and Neural Learning Algorithms , 1989 .

[15]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[16]  Limin Fu.  Integration of neural heuristics into knowledge-based inference , 1989, International 1989 Joint Conference on Neural Networks.

[17]  Prasad Tadepalli,et al.  Lazy ExplanationBased Learning: A Solution to the Intractable Theory Problem , 1989, IJCAI.

[18]  Steve A. Chien.  Using and Refining Simplifications: Explanation-Based Learning of Plans in Intractable Domains , 1989, IJCAI.

[19]  Craig A. Knoblock.  Learning Abstraction Hierarchies for Problem Solving , 1990, AAAI.

[20]  Richard E. Korf,et al.  Real-Time Heuristic Search , 1990, Artif. Intell..

[21]  William W. Cohen.  Learning Approximate Control Rules of High Utility , 1990, ML.

[22]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[23]  Jaime G. Carbonell,et al.  Learning by experimentation: the operator refinement method , 1990 .

[24]  Ryszard S. Michalski,et al.  Machine learning: an artificial intelligence approach volume III , 1990 .

[25]  Thomas G. Dietterich,et al.  Learning with Many Irrelevant Features , 1991, AAAI.

[26]  Katia Sycara,et al.  CADET: a case-based synthesis tool for engineering design , 1991 .

[27]  Saso Dzeroski,et al.  Learning Nonrecursive Definitions of Relations with LINUS , 1991, EWSL.

[28]  Richard S. Sutton,et al.  Planning by Incremental Dynamic Programming , 1991, ML.

[29]  Michael J. Pazzani,et al.  A Knowledge-intensive Approach to Learning Relational Concepts , 1991, ML.

[30]  Oren Etzioni,et al.  DYNAMIC: A New Role for Training Problems in EBL , 1992, ML.

[31]  Gerald DeJong,et al.  COMPOSER: A Probabilistic Solution to the Utility Problem in Speed-Up Learning , 1992, AAAI.

[32]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[33]  Tom Bylander,et al.  Complexity Results for Serial Decomposability , 1992, AAAI.

[34]  Monte Zweben,et al.  Learning to Improve Constraint-Based Scheduling , 1992, Artif. Intell..

[35]  James A. Hendler,et al.  A Validation-Structure-Based Theory of Plan Modification and Reuse , 1992, Artif. Intell..

[36]  Raymond J. Mooney,et al.  Combining FOIL and EBG to Speed-up Logic Programs , 1993, IJCAI.

[37]  Mark A. Peot,et al.  Postponing Threats in Partial-Order Planning , 1993, AAAI.

[38]  Jude W. Shavlik,et al.  Learning Symbolic Rules Using Artificial Neural Networks , 1993, ICML.

[39]  Prasad Tadepalli,et al.  Learning from Queries and Examples with Tree-structured Bias , 1993, ICML.

[40]  Dean A. Pomerleau,et al.  Knowledge-Based Training of Artificial Neural Networks for Autonomous Robot Driving , 1993 .

[41]  Yolanda Gil,et al.  Learning by Experimentation: Incremental Refinement of Incomplete Planning Domains , 1994, International Conference on Machine Learning.

[42]  Steven Minton,et al.  Machine Learning Methods for Planning , 1994 .

[43]  Raymond J. Mooney,et al.  Theory Refinement Combining Analytical and Empirical Methods , 1994, Artif. Intell..

[44]  Thomas G. Dietterich,et al.  Learning Boolean Concepts in the Presence of Many Irrelevant Features , 1994, Artif. Intell..

[45]  Tom Michael Mitchell.  Learning Analytically and Inductively , 1995 .

[46]  Steven Salzberg,et al.  Combining Genetic Algorithms with Memory Based Reasoning , 1995, ICGA.

[47]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[48]  Avrim Blum,et al.  Fast Planning Through Planning Graph Analysis , 1995, IJCAI.

[49]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[50]  Daniel S. Weld,et al.  A Domain-Independent Algorithm for Plan Adaptation , 1994, J. Artif. Intell. Res..

[51]  Kevin D. Ashley,et al.  Reasoning with Reasons in Case-Based Comparisons , 1995, ICCBR.

[52]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[53]  Alberto M. Segre,et al.  The Peaks and Valleys of ALPS: an Adaptive Learning and Planning System for Transportation Scheduling* , 1996 .

[54]  Ralph Bergmann,et al.  On the Role of Abstraction in Case-Based Reasoning , 1996, EWCBR.

[55]  Xuemei Wang,et al.  A Multistrategy Learning System for Planning Operator Acquisition , 1996 .

[56]  Xuemei Wang.  Planning While Learning Operators , 1996, AIPS.

[57]  Lenhart K. Schubert,et al.  Accelerating Partial-Order Planners: Some Techniques for Effective Search Control and Pruning , 1996, J. Artif. Intell. Res..

[58]  Tara A. Estlin,et al.  Multi-Strategy Learning of Search Control for Partial-Order Planning , 1996, AAAI/IAAI, Vol. 1.

[59]  David C. Wilson,et al.  Acquiring Case Adaptation Knowledge: A Hybrid Approach , 1996, AAAI/IAAI, Vol. 1.

[60]  Subbarao Kambhampati,et al.  Failure Driven Dynamic Search Control for Partial Order Planners: An Explanation Based Approach , 1996, Artif. Intell..

[61]  Subbarao Kambhampati,et al.  Design and Implementation of a Replay Framework Based on a Partial Order Planner , 1996, AAAI/IAAI, Vol. 1.

[62]  Pat Langley.  Challenges for the Application of Machine Learning , 1997 .

[63]  Subbarao Kambhampati,et al.  Storing and Indexing Plan Derivations through Explanation-based Analysis of Retrieval Failures , 1997, J. Artif. Intell. Res..

[64]  Bernhard Nebel,et al.  Ignoring Irrelevant Facts and Operators in Plan Generation , 1997, ECP.

[65]  Vincent Aleven,et al.  Reasoning Symbolically About Partially Matched Cases , 1997, IJCAI.

[66]  F. Verdenius,et al.  Proceedings of the workshop Machine Learning Application in the Real World: Methodological Aspects and Implications, hosted by the 14th International Conference on Machine Learning (ICML-97), Nashville, USA , 1997 .

[67]  Pedro Isasi Viñuela,et al.  Genetic Programming and Deductive-Inductive Learning: A Multi-Strategy Approach , 1998, ICML.

[68]  Bart Selman,et al.  The Role of Domain-Specific Knowledge in the Planning as Satisfiability Framework , 1998, AIPS.

[69]  Maria Fox,et al.  The Automatic Inference of State Invariants in TIM , 1998, J. Artif. Intell. Res..

[70]  Ingrid Zukerman,et al.  Inductive Learning of Search Control Rules for Planning , 1998, Artif. Intell..

[71]  Lenhart K. Schubert,et al.  Inferring State Constraints for Domain-Independent Planning , 1998, AAAI/IAAI.

[72]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[73]  Stuart J. Russell,et al.  Bayesian Q-Learning , 1998, AAAI/IAAI.

[74]  Subbarao Kambhampati,et al.  On the Relations Between Intelligent Backtracking and Failure-Driven Explanation-Based Learning in Constraint Satisfaction and Planning , 1998, Artif. Intell..

[75]  Subbarao Kambhampati,et al.  Exploiting Symmetry in the Planning graph via Explanation-Guided Search , 1999, AAAI/IAAI.

[76]  Maria Fox,et al.  The Detection and Exploitation of Symmetry in Planning Problems , 1999, IJCAI.

[77]  Daniel S. Weld,et al.  The LPSAT Engine & Its Application to Resource Planning , 1999, IJCAI.

[78]  Roni Khardon,et al.  Learning Action Strategies for Planning Domains , 1999, Artif. Intell..

[79]  Henry Kautz,et al.  Blackbox: Unifying sat-based and graph-based planning , 1999, International Joint Conference on Artificial Intelligence.

[80]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[81]  Bart Selman,et al.  Unifying SAT-based and Graph-based Planning , 1999, IJCAI.

[82]  Hector Muñoz-Avila,et al.  SHOP: Simple Hierarchical Ordered Planner , 1999, IJCAI.

[83]  David W. Aha,et al.  HICAP: An Interactive Case-Based Planning Architecture and its Application to Noncombatant Evacuation Operations , 1999, AAAI/IAAI.

[84]  Bart Selman,et al.  Learning Declarative Control Rules for Constraint-BAsed Planning , 2000, ICML.

[85]  Pedro M. Domingos,et al.  Version Space Algebra and its Application to Programming by Demonstration , 2000, ICML.

[86]  Subbarao Kambhampati,et al.  Planning Graph as a (Dynamic) CSP: Exploiting EBL, DDB and other CSP Search Techniques in Graphplan , 2000, J. Artif. Intell. Res..

[87]  Ramón García-Martínez,et al.  An Integrated Approach of Learning, Planning, and Execution , 2000, J. Intell. Robotic Syst..

[88]  Hector Geffner,et al.  Learning Generalized Policies in Planning Using Concept Languages , 2000, KR.

[89]  Jussi Rintanen,et al.  An Iterative Algorithm for Synthesizing Invariants , 2000, AAAI/IAAI.

[90]  Paul R. Cohen,et al.  Learning Planning Operators in Real-World, Partially Observable Environments , 2000, AIPS.

[91]  Fahiem Bacchus,et al.  Using temporal logics to express search control knowledge for planning , 2000, Artif. Intell..

[92]  Craig A. Knoblock,et al.  Learning Plan Rewriting Rules , 2000, AIPS.

[93]  Subbarao Kambhampati,et al.  Planning as constraint satisfaction: Solving the planning graph by compiling it into CSP , 2001, Artif. Intell..

[94]  David Maxwell Chickering,et al.  A Bayesian Approach to Tackling Hard Computational Problems (Preliminary Report) , 2001, Electron. Notes Discret. Math..

[95]  Subbarao Kambhampati,et al.  Reviving Partial Order Planning , 2001, IJCAI.

[96]  Terry L. Zimmerman,et al.  Generating parallel plans satisfying multiple criteria in anytime fashion , 2002 .

[97]  Ricardo Aler,et al.  On Control Knowledge Acquisition by Exploiting Human-Computer Interaction , 2002, AIPS.

[98]  Thomas G. Dietterich,et al.  Explanation-Based Learning and Reinforcement Learning: A Unified View , 1995, Machine Learning.

[99]  Tom M. Mitchell,et al.  Explanation-Based Generalization: A Unifying View , 1986, Machine Learning.

[100]  Jack Mostow,et al.  On-Line Learning from Search Failures , 1994, Machine Learning.

[101]  Manuela M. Veloso,et al.  Lazy Incremental Learning of Control Knowledge for Efficiently Obtaining Quality Plans , 1997, Artificial Intelligence Review.

[102]  Francesco Ricci,et al.  Interactive Case-Based Planning for Forest Fire Management , 2000, Applied Intelligence.

[103]  Makoto Haraguchi,et al.  A Goal-Dependent Abstraction for Legal Reasoning by Analogy , 1997, Artificial Intelligence and Law.

[104]  Jürgen Hollatz.  Analogy making in legal reasoning with neural networks and fuzzy logic , 2004, Artificial Intelligence and Law.

[105]  Scott W. Bennett,et al.  Real-world robotics: Learning to plan for robust execution , 1996, Machine Learning.

[106]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[107]  Jaime G. Carbonell,et al.  Derivational Analogy in PRODIGY: Automating Case Acquisition, Storage, and Utilization , 1993, Machine Learning.

[108]  Douglas R. Hofstadter,et al.  Beyond Copycat: Incorporating Self-Watching into a Computer Model of High-Level Perception and Analogy-Making , 2005 .

[109]  Jack Mostow,et al.  On-line learning from search failures , 2005, Machine Learning.

[110]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[111]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[112]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.