Binary classification using ensemble neural networks and interval neutrosophic sets

This paper presents an ensemble neural network and interval neutrosophic sets approach to the problem of binary classification. A bagging technique is applied to an ensemble of pairs of neural networks created to predict degree of truth membership, indeterminacy membership, and false membership values in the interval neutrosophic sets. In our approach, the error and vagueness are quantified in the classification process as well. A number of aggregation techniques are proposed in this paper. We applied our techniques to the classical benchmark problems including ionosphere, pima-Indians diabetes, and liver-disorders from the UCI machine learning repository. Our approaches improve the classification performance as compared to the existing techniques which applied only to the truth membership values. Furthermore, the proposed ensemble techniques also provide better results than those obtained from only a single pair of neural networks.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[3]  H. Guterman,et al.  Knowledge extraction from artificial neural network models , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[4]  Amir F. Atiya,et al.  On the required size of multilayer networks for implementing real-valued functions , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[5]  Padraig Cunningham,et al.  Using Diversity in Preparing Ensembles of Classifiers Based on Different Feature Subsets to Minimize Generalization Error , 2001, ECML.

[6]  P. Smets.  Imperfect information : Imprecision-Uncertainty , 1999 .

[7]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[8]  Stephen Aylward,et al.  An algorithm for neural network architecture generation , 1991 .

[9]  Shuzhi Sam Ge,et al.  An algorithm to determine neural network hidden layer size and weight coefficients , 2000, Proceedings of the 2000 IEEE International Symposium on Intelligent Control. Held jointly with the 8th IEEE Mediterranean Conference on Control and Automation (Cat. No.00CH37147).

[10]  Models of uncertainty in spatial data , 2022 .

[11]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..

[12]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[13]  Zhang-Quan Shen,et al.  Dynamically weighted ensemble neural networks for regression problems , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).

[14]  John F. Kolen,et al.  Backpropagation is Sensitive to Initial Conditions , 1990, Complex Syst..

[15]  Yanqing Zhang,et al.  Interval Neutrosophic Sets and Logic: Theory and Applications in Computing , 2005, ArXiv.

[16]  Lance Chun Che Fung,et al.  Quantification of Uncertainty in Mineral Prospectivity Prediction Using Neural Network Ensembles and Interval Neutrosophic Sets , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[17]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Nitesh V. Chawla,et al.  Bagging-like effects for decision trees and neural nets in protein secondary structure prediction , 2001 .

[19]  Ken-ichi Funahashi,et al.  Multilayer neural networks and Bayes decision theory , 1998, Neural Networks.

[20]  T. Ash,et al.  Dynamic node creation in backpropagation networks , 1989, International 1989 Joint Conference on Neural Networks.

[21]  Matt Duckham,et al.  Uncertainty and geographic information : computational and critical convergence , 2002 .

[22]  Yoh-Han Pao,et al.  Estimation of size of hidden layer on basis of bound of generalization error , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[23]  Petr Hájek,et al.  Fuzzy logic and probability , 1995, UAI.

[24]  Lance Chun Che Fung,et al.  Assessment of Uncertainty in Mineral Prospectivity Prediction Using Interval Neutrosophic Set , 2005, CIS.

[25]  Raymond J. Mooney,et al.  Constructing Diverse Classifier Ensembles using Artificial Training Examples , 2003, IJCAI.

[26]  Panos J. Antsaklis,et al.  A simple method to derive bounds on the size and to train multilayer neural networks , 1991, IEEE Trans. Neural Networks.

[27]  P. Alfeld.  Scattered data interpolation in three or more variables , 1989 .

[28]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[29]  Lance Chun Che Fung,et al.  Mineral Prospectivity Prediction using Interval Neutrosophic Sets , 2006, Artificial Intelligence and Applications.

[30]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[31]  Yoshua Bengio,et al.  Boosting Neural Networks , 2000, Neural Computation.

[32]  D.,et al.  Determining Neural Network Hidden Layer Size using Evolutionary Programming , 2022 .

[33]  Wim Bakker,et al.  Storing and handling vague spatial objects , 2004 .

[34]  Chun Che Fung,et al.  Neural Network Ensembles using Interval Neutrosophic Sets and Bagging for Mineral Prospectivity Prediction and Quantification of Uncertainty , 2006, 2006 IEEE Conference on Cybernetics and Intelligent Systems.

[35]  Rajshekhar Sunderraman,et al.  Interval Neutrosophic Sets , 2004 .

[36]  Isaac Amidror,et al.  Scattered data interpolation methods for electronic imaging systems: a survey , 2002, J. Electronic Imaging.

[37]  Yamashita,et al.  Backpropagation algorithm which varies the number of hidden units , 1989 .