On the Generation of Accurate Predictive Model from Highly Imbalanced Data with Heuristics and Replication Techniques

Recent advancement in the field of life science data mining has inspired researchers and healthcare professionals to apply this novel technology to obtain descriptive patterns and predictive models from biomedical and healthcare databases. The discovery of hidden biomedical patterns from large clinical database can uncover potential knowledge to support prognosis and diagnosis decision makings. However, clinical application of data mining algorithms has a severe problem of low predictive accuracy rate that hampers their wide usage in the clinical environment. We thus focus our study on the improvement of predictive accuracy of the models created from the data mining algorithms. Our main research interest concerns the problem of learning a classification model from a multiclass data set with low prevalence rate of some minority classes. With such data characteristics, directly applying classification data mining techniques such as decision tree induction, regression analysis, neural networks, or support vector machines yields a suboptimal model in terms of predictive accuracy rate. To remedy the imbalanced class distribution among data instances, we apply random over-sampling and synthetic minority over-sampling (SMOTE) techniques to increase the predictive performance of the learned model. In our preliminary study, we consider specific kinds of primary tumors occurring at the frequency rate less than one percent as rare and minority classes. From the experimental results, the SMOTE technique gave a high specificity model, whereas the random over-sampling produced a high sensitivity classifier. The precision performance of a classification model obtained from the random over-sampling technique is on average much better than the model learned from the original imbalanced data set. We then extend our study by designing the heuristic based method to cope with the abundance of irrelevant feature that causes the decrease in learning time and sometimes lower the accuracy rate. The over-sampling technique and the heuristic-based feature selection are coupled as a data preparation method to deal with imbalanced data sets with many irrelevant features. The experimental results on arrhythmia and communities-and-crime data sets show significant improvement on the predicting accuracy, specificity, and sensitivity of the induced models.