An Overview of Classifier Fusion Methods

A number of classifier fusion methods have been recently developed opening an alternative approach leading to a potential improvement in the classification performance. As there is little theory of information fusion itself, currently we are faced with different methods designed for different problems and producing different results. This paper gives an overview of classifier fusion methods and attempts to identify new trends that may dominate this area of research in future. A taxonomy of fusion methods trying to bring some order into the existing “pudding of diversities” is also provided. 1. INTRODUCTION The objective of all decision support systems (DSS) is to create a model, which given a minimum amount of input data/information, is able to produce correct decisions. Quite often, especially in safety critical systems, the correctness of the decisions taken is of crucial importance. In such cases the minimum information constraint is not that important as long as the derivation of the final decision is obtained in a reasonable time. According to one approach, the progress of DSS should be based on continuous development of existing methods as well as discovering new ones. Another approach suggests that as the limits of the existing individual method are approached and it is hard to develop a better one, the solution of the problem might be just to combine existing well performing methods, hoping that better results will be achieved. Such fusion of information seems to be worth applying in terms of uncertainty reduction. Each of individual methods produces some errors, not mentioning that the input information might be corrupted and incomplete. However, different methods performing on different data should produce different errors, and assuming that all individual methods perform well, combination of such multiple experts should reduce overall classification error and as a consequence emphasise correct outputs. Information fusion techniques have been intensively investigated in recent years and their applicability for classification domain has been widely tested [1]-[14]. The problem arouse naturally as a need of improvement of classification rates obtained from individual classifiers. Fusion of data/information can be carried out on three levels of abstraction closely connected with the flow of the classification process: data level fusion, feature level fusion, and classifier fusion [15]. There is little theory about the first two levels of information fusion. However, there have been successful attempts to transform the numerical, interval and linguistic data into a single space of symmetric trapezoidal fuzzy numbers [14], [15], and some heuristic methods have been successfully used for feature level fusion [15]. A number of methods have been developed for classifier fusion also referred to as decision fusion or mixture of experts. Essentially, there are two general groups of classifier fusion techniques. The methods subjectively associated with the first group generally operate on classifiers and put an emphasis on a development of the classifier structure. They do not do anything with classifiers outputs until combination process finds single best classifier or a selected group of classifiers and only then their outputs are taken as a final decision or for further processing [2], [9], [10]. Another group of methods operate mainly on classifiers outputs, and effectively the combination of classifiers outputs is calculated [1], [3]-[8], [11]-[15]. The methods operating on classifiers outputs can be further divided according to the type of the output produced by individual classifiers. A diagrammatic representation of the proposed taxonomy of classifier fusion methods is shown in Figure 1.