A Parallel Radial Basis Probabilistic Neural Network for Scalable Data Mining in Distributed Memory Machines

This work presents scalable algorithms for basic construction of parallel Radial Basis Probabilistic Neural Networks. The final goal is to build a neural network that can efficiently be implemented in distributed memory machines. Thus a fast simple parallel training scheme for RBPNNs is studied, that is based almost solely on Gaussian summations which can by their part be efficiently mapped on parallel as well as on pipeline distributed machines. The suggested training scheme is tested for accuracy and performance and can guarantee simplicity, parallelization and linear speed ups in common parallel implementations, namely neuron parallel and pipelining studied here.

[1]  Haralambos Sarimveis,et al.  A fast training algorithm for RBF networks based on subtractive clustering , 2003, Neurocomputing.

[2]  Lipo Wang,et al.  Data Mining With Computational Intelligence , 2006, IEEE Transactions on Neural Networks.

[3]  Sundaram Suresh,et al.  Parallel implementation of back-propagation algorithm in networks of workstations , 2005, IEEE Transactions on Parallel and Distributed Systems.

[4]  Jon Rigelsford.  Handbook of Neural Network Signal Processing , 2003 .

[5]  Dorit S. Hochbaum,et al.  Approximation Algorithms for NP-Hard Problems , 1996 .

[6]  Jason Weston,et al.  Large-scale kernel machines , 2007 .

[7]  Nikola B. Serbedzija.  Simulating Artificial Neural Networks on Parallel Architectures , 1996, Computer.

[8]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[9]  Margaret H. Dunham,et al.  Data Mining: Introductory and Advanced Topics , 2002 .

[10]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[11]  De-Shuang Huang,et al.  A Constructive Hybrid Structure Optimization Methodology for Radial Basis Probabilistic Neural Networks , 2008, IEEE Transactions on Neural Networks.

[12]  Li Shang,et al.  Palmprint recognition using FastICA algorithm and radial basis probabilistic neural network , 2006, Neurocomputing.

[13]  D.-S. Huang,et al.  Radial Basis Probabilistic Neural Networks: Model and Application , 1999, Int. J. Pattern Recognit. Artif. Intell..

[14]  De-shuang Huang,et al.  The structure optimization of radial basis probabilistic neural networks based on genetic algorithms , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[15]  Wee Ser,et al.  Probabilistic neural-network structure determination for pattern classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[16]  Teofilo F. GONZALEZ,et al.  Clustering to Minimize the Maximum Intercluster Distance , 1985, Theor. Comput. Sci..