Choice effect of linear separability testing methods on constructive neural network algorithms: An empirical study

Several algorithms exist for testing linear separability. The choice of a particular testing algorithm has effects on the performance of constructive neural network algorithms that are based on the transformation of a nonlinear separability classification problem into a linearly separable one. This paper presents an empirical study of these effects in terms of the topology size, the convergence time, and generalisation level of the neural networks. Six different methods for testing linear separability were used in this study. Four out of the six methods are exact methods and the remaining two are approximative ones. A total of nine machine learning benchmarks were used for this study.

[1]  G. Gates,et al.  The reduced nearest neighbor rule (Corresp.) , 1972, IEEE Trans. Inf. Theory.

[2]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[3]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[4]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[5]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[6]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[7]  David A. Elizondo,et al.  The linear separability problem: some testing methods , 2006, IEEE Transactions on Neural Networks.

[8]  Tacha Hicks,et al.  Forensic Interpretation of Glass Evidence , 2000 .

[9]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .

[11]  David A. Elizondo,et al.  The recursive deterministic perceptron neural network , 1998, Neural Networks.

[12]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[13]  David A. Elizondo,et al.  Growing Methods for Constructing Recursive Deterministic Perceptron Neural Networks and Knowledge Extraction , 1998, Artif. Intell..

[14]  Mohamed Tajine,et al.  Adapting the 2-class recursive deterministic perceptron neural network to m-classes , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  P. Diaconis,et al.  Computer-Intensive Methods in Statistics , 1983 .

[17]  T. W. Anderson,et al.  Classification into two Multivariate Normal Distributions with Different Covariance Matrices , 1962 .

[18]  Shaoning Pang,et al.  Face membership authentication using SVM classification tree generated by membership-based LLE data partition , 2005, IEEE Trans. Neural Networks.

[19]  Belur V. Dasarathy,et al.  Nosing Around the Neighborhood: A New System Structure and Classification Rule for Recognition in Partially Exposed Environments , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[21]  Václav Hlavác,et al.  Ten Lectures on Statistical and Structural Pattern Recognition , 2002, Computational Imaging and Vision.

[22]  Ivan Bratko,et al.  ASSISTANT 86: A Knowledge-Elicitation Tool for Sophisticated Users , 1987, EWSL.

[23]  J. Stoer,et al.  Convexity and Optimization in Finite Dimensions I , 1970 .

[24]  Eugenius Kaszkurewicz,et al.  Solving systems of linear equations via gradient systems with discontinuous righthand sides: application to LS-SVM , 2005, IEEE Transactions on Neural Networks.

[25]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[26]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[27]  David A. Elizondo,et al.  Current Trends on Knowledge Extraction and Neural Networks , 2005, ICANN.

[28]  Sebastian Thrun,et al.  The MONK''s Problems-A Performance Comparison of Different Learning Algorithms, CMU-CS-91-197, Sch , 1991 .

[29]  G. Gates.  The Reduced Nearest Neighbor Rule , 1998 .