An efficient space division–based width optimization method for RBF network using fuzzy clustering algorithms

The Radial Basis Function (RBF) network is one of the most widely used surrogate models in multidisciplinary design optimization. However, one of the challenges of applying the RBF network to engineering problems is how to efficiently optimize its width parameters. In this work, a novel space division–based width optimization (SDWO) method is proposed to decompose the complex multivariable width optimization into several small-scale single-variable width optimizations. The SDWO method consists of two main steps. First, a two-stage fuzzy clustering algorithm is carried out to group the samples and divide the input space into several overlapping local regions, and the overlapping degree is controlled by a dimensionless expansion factor. Second, in each local region, one small-scale local RBF network (LRBFN) is constructed by solving a single-variable optimization problem when the LRBFN is needed for prediction. All these LRBFNs are independent of each other and can be constructed in parallel. The proposed method is efficient and particularly suitable for large sample sets. Test results of four sample sets verify that the proposed SDWO method has better performance than the conventional width optimization method in terms of both training efficiency and approximation accuracy. An inter-stage structure optimization is carried out, which demonstrates the efficiency of the proposed method in practical engineering applications.

[1]  Stephen J. Leary,et al.  A parallel updating scheme for approximating and optimizing high fidelity computer simulations , 2004 .

[2]  Dave Watson,et al.  Spatial tessellations: concepts and applications of voronoi diagrams: by Atsuyuki Okabe, Barry Boots, and Kokichi Sugihara, 1992, John Wiley & Sons, New York, 532 p., ISBN 0 471 93430 5, US $112.00 , 1993 .

[3]  Najdan Vukovic,et al.  A growing and pruning sequential learning algorithm of hyper basis function neural network for function approximation , 2013, Neural Networks.

[4]  Zhao Jing,et al.  A cooperative radial basis function method for variable-fidelity surrogate modeling , 2017 .

[5]  Xu Li,et al.  A surrogate model based nested optimization framework for inverse problem considering interval uncertainty , 2018 .

[6]  Mark J. L. Orr.  Optimising the widths of radial basis functions , 1998, Proceedings 5th Brazilian Symposium on Neural Networks (Cat. No.98EX209).

[7]  Haitao Liu,et al.  A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design , 2017, Structural and Multidisciplinary Optimization.

[8]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[9]  Anirban Chaudhuri,et al.  Parallel surrogate-assisted global optimization with expensive functions – a survey , 2016 .

[10]  Michel Verleysen,et al.  On the Kernel Widths in Radial-Basis Function Networks , 2003, Neural Processing Letters.

[11]  Teng Long,et al.  Multidisciplinary modeling and surrogate assisted optimization for satellite constellation systems , 2018, Structural and Multidisciplinary Optimization.

[12]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[13]  Teng Long,et al.  Sequential RBF surrogate-based efficient optimization method for engineering design problems with expensive black-box functions , 2014 .

[14]  Edwin R. van Dam,et al.  Bounds for Maximin Latin Hypercube Designs , 2007, Oper. Res..

[15]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[17]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[18]  Xiao-Jun Zeng,et al.  Fuzzy C-means++: Fuzzy C-means with effective seeding initialization , 2015, Expert Syst. Appl..

[19]  Pierre Hansen,et al.  Cluster analysis and mathematical programming , 1997, Math. Program..

[20]  Qingfu Zhang,et al.  A multiobjective optimization based framework to balance the global exploration and local exploitation in expensive optimization , 2015, J. Glob. Optim..

[21]  Christian W. Dawson,et al.  A review of genetic algorithms applied to training radial basis function networks , 2004, Neural Computing & Applications.

[22]  De-Shuang Huang,et al.  A Hybrid Forward Algorithm for RBF Neural Network Construction , 2006, IEEE Transactions on Neural Networks.

[23]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[24]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[25]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[26]  Yong Zhao,et al.  Concurrent Subspace Width Optimization Method for RBF Neural Network Modeling , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Maria Bortman,et al.  A Growing and Pruning Method for Radial Basis Function Networks , 2009, IEEE Transactions on Neural Networks.

[28]  Sung-Kwun Oh,et al.  Design of K-means clustering-based polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution , 2012, Neurocomputing.

[29]  Haralambos Sarimveis,et al.  Radial Basis Function Network Training Using a Nonsymmetric Partition of the Input Space and Particle Swarm Optimization , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[31]  Michel Verleysen,et al.  Width optimization of the Gaussian kernels in Radial Basis Function Networks , 2002, ESANN.

[32]  J. A. Leonard,et al.  Radial basis function networks for classifying process faults , 1991, IEEE Control Systems.

[33]  M. Tanemura,et al.  Voronoi diagram description of the maternal surface of the placenta: Preliminary report , 2011, The journal of obstetrics and gynaecology research.

[34]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .

[35]  Taimoor Akhtar,et al.  Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection , 2016, J. Glob. Optim..

[36]  Tzong-Jer Chen,et al.  Fuzzy c-means clustering with spatial information for image segmentation , 2006, Comput. Medical Imaging Graph..

[37]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[38]  Jouni Lampinen,et al.  A differential evolution based incremental training method for RBF networks , 2005, GECCO '05.

[39]  Václav Skala,et al.  Large scattered data interpolation with radial basis functions and space subdivision , 2017, Integr. Comput. Aided Eng..

[40]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[41]  Alaa F. Sheta,et al.  Time-series forecasting using GA-tuned radial basis functions , 2001, Inf. Sci..

[42]  D. Lowe,et al.  Adaptive radial basis function nonlinearities, and the problem of generalisation , 1989 .

[43]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[44]  Roman Neruda,et al.  Learning methods for radial basis function networks , 2005, Future Gener. Comput. Syst..

[45]  Dimosthenis Kyriazis,et al.  Hierarchical Fuzzy Clustering in Conjunction with Particle Swarm Optimization to Efficiently Design RBF Neural Networks , 2014, Journal of Intelligent & Robotic Systems.

[46]  Jianbin Du,et al.  Hybrid optimization of a vibration isolation system considering layout of structure and locations of components , 2017 .

[47]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[48]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Combining RBF Networks Trained by Different Clustering Techniques , 2001, Neural Processing Letters.