Measuring Plan Diversity: Pathologies in Existing Approaches and A New Plan Distance Metric

In this paper we present a plan-plan distance metric based on Kolmogorov (Algorithmic) complexity. Generating diverse sets of plans is useful for tasks such as probing user preferences and reasoning about vulnerability to cyber attacks. Generating diverse plans, and comparing different diverse planning approaches requires a domain-independent, theoretically motivated definition of the diversity distance between plans. Previously proposed diversity measures are not theoretically motivated, and can provide inconsistent results on the same plans. We define the diversity of plans in terms of how surprising one plan is given another or, its inverse, the conditional information in one plan given another. Kolmogorov complexity provides a domain independent theory of conditional information. While Kolmogorov complexity is not computable, a related metric, Normalized Compression Distance (NCD), provides a well-behaved approximation. In this paper we introduce NCD as an alternative diversity metric, and analyze its performance empirically, in comparison with previous diversity measures, showing strengths and weaknesses of each. We also examine the use of different compressors in NCD. We show how NCD can be used to select a training set for HTN learning, giving an example of the utility of diversity metrics. We conclude with suggestions for future work on improving, extending, and applying it to serve new applications.