MultiLearner Based Recursive Supervised Training

In supervised learning, most single solution neural networks such as Constructive Backpropagation give good results when used with some datasets but not with others. Others such as Probabilistic Neural Networks (PNN) fit a curve to perfection but need to be manually tuned in the case of noisy data. Recursive Percentage based Hybrid Pattern Training (RPHP) overcomes this problem by recursively training subsets of the data, thereby using several neural networks. MultiLearner based Recursive Training (MLRT) is an extension of this approach, where a combination of existing and new learners are used and subsets are trained using the weak learner which is best suited for this subset. We observed that empirically, MLRT performs considerably well as compared to RPHP and other systems on benchmark data with 11% improvement in accuracy on the spam dataset and comparable performances on the vowel and the two-spiral problems.

[1]  David M. Mount,et al.  The analysis of a simple k-means clustering algorithm , 2000, SCG '00.

[2]  Kiruthika Ramanathan,et al.  Recursive percentage based hybrid pattern (RPHP) training for curve fitting , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[3]  George Karypis,et al.  A Comparison of Document Clustering Techniques , 2000 .

[4]  T. Ash,et al.  Dynamic node creation in backpropagation networks , 1989, International 1989 Joint Conference on Neural Networks.

[5]  Esa Alhoniemi,et al.  Clustering of the self-organizing map , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  Ana L. N. Fred,et al.  Combining multiple clusterings using evidence accumulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[10]  Miin-Shen Yang,et al.  Alternative c-means clustering algorithms , 2002, Pattern Recognit..

[11]  Timur Ash,et al.  Dynamic node creation in backpropagation networks , 1989 .

[12]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[13]  Anil K. Jain,et al.  Adaptive clustering ensembles , 2004, ICPR 2004.

[14]  Chunyu Bao,et al.  Task Decomposition Using Pattern Distributor , 2004 .

[15]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[16]  David J. Montana,et al.  A Weighted Probabilistic Neural Network , 1991, NIPS.

[17]  Mikko Lehtokangas.  Modelling with constructive backpropagation , 1999, Neural Networks.

[18]  Arthur Flexer,et al.  On the use of self-organizing maps for clustering and visualization , 1999, Intell. Data Anal..

[19]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[20]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.