Select Objective Functions for Multiple Criteria Programming Classification

Existing supervised learning models are generally built upon the basis of only one single objective function, through the minimizing of the square-loss (neural networks) or the minimizing of the information entropy (decision tree). Due to the inherent complexity of the real-life data, learning models merely based on only one single objective function are always inadequate. Consequently, many well-known classification models adopt multiple objective optimization to guide the learning process. For example, Fisherpsilas linear discriminant analysis (LDA) is built by maximizing the ldquobetween-class variancerdquo (the first objective function) and minimizing the ldquowithin-class variancerdquo (the second objective function); SVM is built by maximizing the ldquomarginal distancerdquo (the first objective function) and minimizing the ldquoerror distancerdquo (the second objective function). In this paper, we combine Fisherpsilas LDA (maximizing the ldquobetween-class variancerdquo) measure and SVMpsilas minimizing the ldquoerror distancerdquo measure to formulate a new multiple objective classification model, namely minimal error and maximal between-class variance (MEMBV) model. Experimental results demonstrate the performance of the proposed new model on synthetic and real-life datasets.