A Linear-Bayes Classifier

Naive Bayes is a well known and studied algorithm both in statistics and machine learning. Although its limitations with respect to expressive power, this procedure has a surprisingly good performance in a wide variety of domains, including many where there are clear dependencies between attributes. In this paper we address its main perceived limitation - its inability to deal with attribute dependencies. We present Linear Bayes that uses, for the continuous attributes, a multivariate normal distribution to compute the require probabilities. In this way, the interdependencies between the continuous attributes are considered. On the empirical evaluation, we compare Linear Bayes against a naive-Bayes that discretize continuous attributes, a naive-Bayes that assumes a univariate Gaussian for continuous attributes, and a standard Linear discriminant function. We show that Linear Bayes is a plausible algorithm, that competes quite well against other well established techniques.