Cost-sensitive business failure prediction when misclassification costs are uncertain: A heterogeneous ensemble selection approach

Abstract In order to assess risks associated with establishing relationships with corporate partners such as clients, suppliers, debtors or contractors, decision makers often turn to business failure prediction models. While a large body of literature has focused on optimizing and evaluating novel methods in terms of classification accuracy, recent research has acknowledged the existence of asymmetric misclassification costs associated with prediction errors and thus, advocates the usage of alternative evaluation metrics. However, these papers often assume a misclassification cost matrix to be known and fixed for both the training and the evaluation of models, whereas in reality these costs are often uncertain. This paper presents a methodological framework based upon heterogeneous ensemble selection and multi-objective optimization for cost-sensitive business failure prediction that accommodates uncertainty at the level of misclassification costs. The framework assumes unknown costs during model training and accommodates varying degrees of uncertainty during model deployment. Specifically, NSGA-II is deployed to optimize cost space resulting in a set of pareto-optimal ensemble classifiers where every learner minimizes expected misclassification cost for a specific range of cost ratios. An extensive set of experiments evaluates the method on multiple data sets and for different scenarios that reflect the extent to which cost ratios are known during model deployment. Results clearly demonstrate the ability of our method to minimize cost under the absence of exact knowledge of misclassification costs.