Faster RBF Network Learning Utilizing Singular Regions

There are two ways to learn radial basis function (RBF) networks: one-stage and two-stage learnings. Recently a very powerful one-stage learning method called RBF-SSF has been proposed, which can stably find a series of excellent solutions, making good use of singular regions, and can monotonically decrease training error along with the increase of hidden units. RBF-SSF was built by applying the SSF (singularity stairs following) paradigm to RBF networks; the SSF paradigm was originally and successfully proposed for multilayer perceptrons. Although RBF-SSF has the strong capability to find excellent solutions, it required a lot of time mainly because it computes the Hessian. This paper proposes a faster version of RBF-SSF called RBF-SSF(pH) by introducing partial calculation of the Hessian. The experiments using two datasets showed RBF-SSF(pH) ran as fast as usual one-stage learning methods while keeping the excellent solution quality.

[1]  Max A. Little,et al.  Exploiting Nonlinear Recurrence and Fractal Scaling Properties for Voice Disorder Detection , 2007 .

[2]  Hui Wang,et al.  Using Radial Basis Function Networks for Function Approximation and Classification , 2012 .

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  Tohru Nitta,et al.  Local minima in hierarchical structures of complex-valued neural networks , 2013, Neural Networks.

[5]  R. Nakano,et al.  Multilayer Perceptron Learning Utilizing Singular Regions and Search Pruning , 2013 .

[6]  Marcelino Lázaro,et al.  A new EM-based training algorithm for RBF networks , 2003, Neural Networks.

[7]  Ryohei Nakano,et al.  A New Method for Learning RBF Networks by Utilizing Singular Regions , 2018, ICAISC.

[8]  Ryohei Nakano,et al.  A yet faster version of complex-valued multilayer perceptron learning using singular regions and search pruning , 2015, 2015 7th International Joint Conference on Computational Intelligence (IJCCI).

[9]  Friedhelm Schwenker,et al.  Three learning phases for radial-basis-function networks , 2001, Neural Networks.

[10]  Kenji Fukumizu,et al.  Local minima and plateaus in hierarchical structures of multilayer perceptrons , 2000, Neural Networks.

[11]  Reuven Y. Rubinstein,et al.  Optimization of computer simulation models with rare events , 1997 .

[12]  Max A. Little,et al.  Exploiting Nonlinear Recurrence and Fractal Scaling Properties for Voice Disorder Detection , 2007, Biomedical engineering online.