Robust Approach for Estimating Probabilities in Naive-Bayes Classifier

Naive-Bayes classifier is a popular technique of classification in machine learning. Improving the accuracy of naive-Bayes classifier will be significant as it has great importance in classification using numerical attributes. For numeric attributes, the conditional probabilities are either modeled by some continuous probability distribution over the range of that attribute's values or by conversion of numeric attribute to discrete one using discretization. The limitation of the classifier using discretization is that it does not classify those instances for which conditional probabilities of any of the attribute value for every class is zero. The proposed method resolves this limitation of estimating probabilities in the naive-Bayes classifier and improve the classification accuracy for noisy data. The proposed method is efficient and robust in estimating probabilities in the naive-Bayes classifier. The proposed method has been tested over a number of databases of UCI machine learning repository and the comparative results of existing naive-Bayes classifier and proposed method has also been illustrated.

[1]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[2]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[3]  Geoffrey I. Webb,et al.  Incremental Discretization for Naïve-Bayes Classifier , 2006, ADMA.

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[6]  Jason Catlett,et al.  On Changing Continuous Attributes into Ordered Discrete Attributes , 1991, EWSL.

[7]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[8]  Ying Yang,et al.  A comparative study of discretization methods for naive-Bayes classifiers , 2002 .

[9]  Bojan Cestnik,et al.  Estimating Probabilities: A Crucial Task in Machine Learning , 1990, ECAI.

[10]  Pat Langley,et al.  An Analysis of Bayesian Classifiers , 1992, AAAI.

[11]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[12]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[13]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[14]  Randy Kerber,et al.  ChiMerge: Discretization of Numeric Attributes , 1992, AAAI.

[15]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[16]  Geoffrey I. Webb,et al.  On Why Discretization Works for Naive-Bayes Classifiers , 2003, Australian Conference on Artificial Intelligence.