Using FALCES against bias in automated decisions by integrating fairness in dynamic model ensembles

As regularly reported in the media, automated classifications and decisions based on machine learning models can cause unfair treatment of certain groups of a general population. Classically, the machine learning models are designed to make highly accurate decisions in general. When one machine learning model is not sufficient to define the possibly complex boundary between classes, multiple “specialized” models are used within a model ensemble to further boost accuracy. In particular, dynamic model ensembles pick the most accurate model for each query object, by applying the model that performed best on similar data. Given the labeled data on which models are trained, it is not surprising that any bias possibly present in the data will reflect in the classifiers using the models. To mitigate this, recent work has proposed fair model ensembles, that instead of focusing (solely) on accuracy also optimize global fairness, which is quantified using bias metrics. However, such global fairness that globally minimizes bias may exhibit imbalances in different regions of the data, e.g., caused by some local bias maxima leading to local unfairness. In this paper, we propose to bridge the gap between dynamic model ensembles and fair model ensembles and investigate the problem of devising locally fair and accurate dynamic model ensembles, which ultimately optimize for equal opportunity of similar subjects. Our evaluation shows that our approach outperforms the state-of-the-art for different types and degrees of bias present in training data in terms of both local and global fairness, while reaching comparable accuracy.