Increasing the Performance of a Training Algorithm for Local Model Networks

In this paper the improvement of the established training algorithm HILOMOT is presented. HILOMOT is a hierarchical tree-construction method based on the ideas of neuronal networks and fuzzy-systems and an advancement of the wellknown LOLIMOT-Algorithm. During the training the input space is divided into subregions with the help of validity functions. This is done iteratively by splitting the current worst local model, until an specified exit condition is reached. For every subregion one local linear model is estimated by a weighted least squares method. The main purpose is keeping the number of local models low. Therefore the split position is optimized during the training. The optimization problem is nonlinear, so a gradient-based nonlinear local optimization method, called Quasi-Newton method, is used. The main drawback of this approach is its long calculation time. The most time consuming part is the numerical calculation of the gradient done by the finite difference technique.To avoid this problem the numerical approach is replaced by the analytical gradient. This leads to a significant reduction of the training time without decreasing the approximation quality.