Comparison of performance of variants of single-layer perceptron algorithms on nonseparable data

We present a detailed experimental comparison of the pocket algorithm thermal percep tron and barycentric correction procedure algorithms that most commonly used algorithms for training threshold logic units TLUs Each of these algorithms represent stable variants of the standard perceptron learning rule in that they guarantee convergence to zero classi cation errors on datasets that are linearly separable and attempt to classify as large a subset of the training patterns as possible for datasets that are not linearly separable For datasets involving patterns distributed amongM di erent categories M a group of M TLUs is trained one for each of the output classes These TLU s can be trained either independently or as a winner take all WTA group The latter mechanism accounts for the interactions among the di erent output classes and exploits the fact that a pattern can ideally belong to only one of the M output classes The extension of the pocket algorithm to the WTA output strategy is direct In this paper we present heuristic extensions of the thermal perceptron and the barycentric correction procedure to WTA groups and empirically verify their performance The performance of these algorithms was measured in a collection of carefully chosen benchmarks datasets We report the training and generalization accuracies of these algorithms on the di erent datasets along with the learning time in seconds In addition a comparison of the learning speeds of the al gorithms is indicated by means of learning curve plots on two datasets We identify and report some distinguishing traits of these algorithms which could possibly enable making an informed choice of the training algorithm combined with constructive learning algorithms when certain characteristics of the dataset are known