Combining localized fusion and dynamic selection for high-performance SVM

We present multiple SVMs combined by localized fusion and dynamic selection.Clustering OVA SVMs trained for each class constructs multiple decision templates.A naive Bayes classifier selects the localized models dynamically for a new sample.Experiments on nine benchmark datasets show its superiority against alternatives.It proves to effectively manage the unbiased-variance and bias in real datasets. To resolve class-ambiguity in real world problems, we previously presented two different ensemble approaches with support vector machines (SVMs): multiple decision templates (MuDTs) and dynamic ordering of one-vs.-all SVMs (DO-SVMs). MuDTs is a classifier fusion method, which models intra-class variations as subclass templates. On the other hand, DO-SVMs is an ensemble method that dynamically selects proper SVMs to classify an input sample based on its class probability. In this paper, we newly propose a hybrid scheme of those two approaches to utilize their complementary properties. The localized fusion approach of MuDTs increases variance of the classification models while the dynamic selection scheme of DO-SVMs reduces the unbiased-variance, which causes incorrect prediction. We show the complementary properties of MuDTs and DO-SVMs with several benchmark datasets and verify the performance of the proposed method. We also test how much our method could increase its baseline accuracy by comparing with other combinatorial ensemble approaches.