The Naïve Associative Classifier (NAC): A novel, simple, transparent, and accurate classification model evaluated on financial data

Abstract In this paper the Naive Associative Classifier (NAC), a novel supervised learning model, is presented. Its strengths lie in its simplicity, transparency, transportability and accuracy. The creation, design, implementation and application of the NAC are sustained by an original similarity operator of our own design, the Mixed and Incomplete Data Similarity Operator (MIDSO). One of the key features of MIDSO is its ability to handle missing values as well as mixed numerical and categorical data types. The proposed model was tested by performing numerical experiments using finance-related datasets including credit assignment, bank telemarketing, bankruptcy, and banknote authentication. The experimental results show the adequacy of the model for decision support in those environments, outperforming several state-of-the-art pattern classifiers. Additionally, the advantages and limitations of the NAC, as well as possible improvements, are discussed.

[1]  Carlos F.M. Coimbra,et al.  Nearest-neighbor methodology for prediction of intra-hour global horizontal and direct normal irradiances , 2015 .

[2]  Safdar Ali,et al.  Prediction of human breast and colon cancers from imbalanced data using nearest neighbor and support vector machines , 2014, Comput. Methods Programs Biomed..

[3]  Francisco Herrera,et al.  An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics , 2013, Inf. Sci..

[4]  Aruna Tiwari,et al.  Breast cancer diagnosis using Genetically Optimized Neural Network model , 2015, Expert Syst. Appl..

[5]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[6]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[7]  I. Roman-Godinez,et al.  A New Classifier Based on Associative Memories , 2006, 2006 15th International Conference on Computing.

[8]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[9]  Cornelio Yáñez-Márquez,et al.  Data Stream Classification Based on the Gamma Classifier , 2015 .

[10]  David C. Yen,et al.  A comparative study of classifier ensembles for bankruptcy prediction , 2014, Appl. Soft Comput..

[11]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[12]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[13]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[14]  Baoping Tang,et al.  Fault diagnosis method based on incremental enhanced supervised locally linear embedding and adaptive nearest neighbor classifier , 2014 .

[15]  David West,et al.  Neural network ensemble strategies for financial decision applications , 2005, Comput. Oper. Res..

[16]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[17]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[18]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[19]  M.N.S. Swamy,et al.  Neural Networks and Statistical Learning , 2013 .

[20]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[21]  David G. Stork,et al.  Pattern Classification , 1973 .

[22]  Taskin Kavzoglu,et al.  The use of logistic model tree (LMT) for pixel- and object-based classifications using high-resolution WorldView-2 imagery , 2017 .

[23]  D. Wolpert.  The Supervised Learning No-Free-Lunch Theorems , 2002 .

[24]  Hojjat Adeli,et al.  Computer-Aided Diagnosis of Parkinson’s Disease Using Enhanced Probabilistic Neural Network , 2015, Journal of Medical Systems.

[25]  José Salvador Sánchez,et al.  On the suitability of resampling techniques for the class imbalance problem in credit scoring , 2013, J. Oper. Res. Soc..

[26]  Stan Szpakowicz,et al.  Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation , 2006, Australian Conference on Artificial Intelligence.

[27]  G. Arturo Sanchez-Azofeifa,et al.  Mapping Tropical Dry Forest Succession With CHRIS/PROBA Hyperspectral Images Using Nonparametric Decision Trees , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[28]  J. P. Marques de Sá,et al.  Pattern Recognition: Concepts, Methods and Applications , 2001 .

[29]  Byeong Ho Kang,et al.  Investigation and improvement of multi-layer perception neural networks for credit scoring , 2015, Expert Syst. Appl..

[30]  Mikel Galar,et al.  Analysing the classification of imbalanced data-sets with multiple classes: Binarization techniques and ad-hoc approaches , 2013, Knowl. Based Syst..

[31]  David L. Olson,et al.  Comparative analysis of data mining methods for bankruptcy prediction , 2012, Decis. Support Syst..

[32]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[33]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[34]  Cornelio Yáñez-Márquez,et al.  A novel associative model for time series data mining , 2014, Pattern Recognit. Lett..

[35]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[36]  Yanchun Zhang,et al.  Epileptic seizure detection from EEG signals using logistic model trees , 2016, Brain Informatics.

[37]  Oscar Camacho Nieto,et al.  Pollutants Time-Series Prediction using the Gamma Classifier , 2011, Int. J. Comput. Intell. Syst..

[38]  Taravat Ghafourian,et al.  Decision trees to characterise the roles of permeability and solubility on the prediction of oral absorption. , 2015, European journal of medicinal chemistry.

[39]  Paulo Cortez,et al.  A data-driven approach to predict the success of bank telemarketing , 2014, Decis. Support Syst..

[40]  Itzamá López-Yáñez,et al.  Instance Selection in the Performance of Gamma Associative Classifier , 2015, Res. Comput. Sci..

[41]  Cornelio Yáñez-Márquez,et al.  A New Model of BAM: Alpha-Beta Bidirectional Associative Memories , 2007, J. Comput..

[42]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[43]  J. Ruiz-Shulcloper,et al.  Pattern recognition with mixed and incomplete data , 2008, Pattern Recognition and Image Analysis.

[44]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[45]  Josef Kittler,et al.  Inverse random under sampling for class imbalance problem and its application to multi-label classification , 2012, Pattern Recognit..

[46]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[47]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[48]  Ingoo Han,et al.  The discovery of experts' decision rules from qualitative bankruptcy data using genetic algorithms , 2003, Expert Syst. Appl..

[49]  Eibe Frank,et al.  Logistic Model Trees , 2003, Machine Learning.

[50]  Baojun Zhao,et al.  Compressed-Domain Ship Detection on Spaceborne Optical Image Using Deep Neural Network and Extreme Learning Machine , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[51]  José Francisco Martínez Trinidad,et al.  Editing and Training for ALVOT, an Evolutionary Approach , 2003, IDEAL.

[52]  Cornelio Yáñez-Márquez,et al.  One-Hot Vector Hybrid Associative Classifier for Medical Data Classification , 2014, PloS one.

[53]  Oscar Camacho Nieto,et al.  An associative memory approach to medical decision support systems , 2012, Comput. Methods Programs Biomed..

[54]  Yenny Villuendas,et al.  Evolutive Improvement of Parameters in an Associative Classifier , 2015, IEEE Latin America Transactions.

[55]  Abdulhamit Subasi,et al.  Comparison of decision tree algorithms for EMG signal classification using DWT , 2015, Biomed. Signal Process. Control..

[56]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[57]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[58]  Oscar Camacho Nieto,et al.  Clasificador de Heaviside , 2015 .

[59]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[60]  J. Seibert,et al.  Flood‐type classification in mountainous catchments using crisp and fuzzy decision trees , 2015 .

[61]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .