Learning Certifiably Optimal Rule Lists: A Case for Discrete Optimization in the 21st Century

We demonstrate a new algorithm, CORELS, for constructing rule lists. It finds the optimal rule list and produces proof of that optimality. Rule lists, which are lists composed of if-then statements, are similar to decision trees and are useful because each step in the model’s decision making process is understandable by humans. CORELS uses the discrete optimization technique of branch-and-bound to eliminate large parts of the search space and turn this into a computationally feasible problem. We use three types of bounds: bounds inherent to the rules themselves, bounds based on the current best solution, and bounds based on symmetries between rule lists. In addition, we use efficient data structures to minimize the memory usage and runtime of our algorithm on this exponentially difficult problem. Our algorithm demonstrates the feasibility of finding optimal solutions in a search space using discrete optimization on modern computers. Our algorithm therefore allows for the discovery and analysis of optimal solutions to problems requiring human-interpretable algorithms.