The Proper Treatment of Case-Based Planning.

One of the major obstacles to progress in Artificial Intelligence (AI) is the absence of an adequate methodology; in particular, a methodology for evaluation the fragmentary systems which make up much of the corpus of AI research. This paper proposes a novel evaluation methodology which is designed to assess planning systems relative to an ideal system. The focus is on case-based planners because their diversity presents a considerable challenge to this form of ideal evauation. Two main priciples about the costs incurred in case-based planning systems are proposed; one which deals with the cost of knowledge acquisition and one with the cost of computation. We then use these principles to define the ideal case-based planning system and develop precise metrics that can be applied to existing systems in order to determine how far they are from this ideal. These metrics are applied to two case-based planners in the literature in order to illustrate their ease of application. Finally, the importance and implications of this type of evaluation for case-based planning, and AI in general are considered.