Variable Selection in Nonlinear Modeling Based on RBF Networks and Evolutionary Computation

In this paper a novel variable selection method based on Radial Basis Function (RBF) neural networks and genetic algorithms is presented. The fuzzy means algorithm is utilized as the training method for the RBF networks, due to its inherent speed, the deterministic approach of selecting the hidden node centers and the fact that it involves only a single tuning parameter. The trade-off between the accuracy and parsimony of the produced model is handled by using Final Prediction Error criterion, based on the RBF training and validation errors, as a fitness function of the proposed genetic algorithm. The tuning parameter required by the fuzzy means algorithm is treated as a free variable by the genetic algorithm. The proposed method was tested in benchmark data sets stemming from the scientific communities of time-series prediction and medicinal chemistry and produced promising results.

[1]  Hojjat Adeli,et al.  Bilevel Parallel Genetic Algorithms for Optimization of Large Steel Structures , 2001 .

[2]  K. Baumann,et al.  A systematic evaluation of the benefits and hazards of variable selection in latent variable regression. Part I. Search algorithm, theory and simulations , 2002 .

[3]  John H. Kalivas,et al.  Further investigation on a comparative study of simulated annealing and genetic algorithm for wavelength selection , 1995 .

[4]  Xin Wang,et al.  A Fast Method for Implicit Surface Reconstruction Based on Radial Basis Functions Network from 3D Scattered Points , 2007, Int. J. Neural Syst..

[5]  Chris L. Waller,et al.  Development and Validation of a Novel Variable Selection Technique with Application to Multidimensional Quantitative Structure-Activity Relationship Studies , 1999, J. Chem. Inf. Comput. Sci..

[6]  Hojjat Adeli,et al.  Resource Scheduling Using Neural Dynamics Model of Adeli and Park , 2001 .

[7]  Kay W. Axhausen,et al.  Optimization of Large Transport Networks Using the Ant Colony Heuristic , 2009, Comput. Aided Civ. Infrastructure Eng..

[8]  Hojjat Adeli,et al.  Dynamic fuzzy wavelet neuroemulator for non‐linear control of irregular building structures , 2008 .

[9]  C. L. Mallows.  Some comments on C_p , 1973 .

[10]  Sabine Van Huffel,et al.  Preoperative prediction of malignancy of ovarian tumors using least squares support vector machines , 2003, Artif. Intell. Medicine.

[11]  Günter Gauglitz,et al.  Genetic algorithms and neural networks for the quantitative analysis of ternary mixtures using surface plasmon resonance , 2003 .

[12]  Douglas B. Kell,et al.  Genetic algorithms as a method for variable selection in multiple linear regression and partial least squares regression, with applications to pyrolysis mass spectrometry , 1997 .

[13]  Hojjat Adeli,et al.  Optimization of space structures by neural dynamics , 1995, Neural Networks.

[14]  D. Livingstone,et al.  Structure-activity relationships of antifilarial antimycin analogues: a multivariate pattern recognition study. , 1990, Journal of medicinal chemistry.

[15]  Alexandru Stancu,et al.  Nonlinear System Identification Based on Internal Recurrent Neural Networks , 2009, Int. J. Neural Syst..

[16]  H. Akaike.  A new look at the statistical model identification , 1974 .

[17]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[18]  James A. Reggia,et al.  A distributed learning algorithm for particle systems , 2009, Integr. Comput. Aided Eng..

[19]  Yonggwan Won,et al.  An Improvement of Extreme Learning Machine for Compact Single-Hidden-Layer Feedforward Neural Networks , 2008, Int. J. Neural Syst..

[20]  Yinghua Lin,et al.  A new approach to fuzzy-neural system modeling , 1995, IEEE Trans. Fuzzy Syst..

[21]  Witold Pedrycz,et al.  Experience-Consistent Modeling for Radial Basis Function Neural Networks , 2008, Int. J. Neural Syst..

[22]  Brigitte Charnomordic,et al.  Knowledge discovery for control purposes in food industry databases , 2001, Fuzzy Sets Syst..

[23]  K. Asai,et al.  Fuzzy linear programming problems with fuzzy numbers , 1984 .

[24]  Hojjat Adeli,et al.  A probabilistic neural network for earthquake magnitude prediction , 2009, Neural Networks.

[25]  Christophe Charrier,et al.  International Journal of Neural Systems Special Issue on Issue's Topic C World Scientific Publishing Company Tabu Search Model Selection for Svm , 2022 .

[26]  António Pais Antunes,et al.  An Efficient Simulated Annealing Algorithm for Regional Wastewater System Planning , 2009, Comput. Aided Civ. Infrastructure Eng..

[27]  Maria Q. Feng,et al.  Long-Term Monitoring and Identification of Bridge Structural Parameters , 2009 .

[28]  Hojjat Adeli,et al.  Principal Component Analysis-Enhanced Cosine Radial Basis Function Neural Network for Robust Epilepsy and Seizure Detection , 2008, IEEE Transactions on Biomedical Engineering.

[29]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[30]  Paul Schonfeld,et al.  Prescreening and Repairing in a Genetic Algorithm for Highway Alignment Optimization , 2009, Comput. Aided Civ. Infrastructure Eng..

[31]  Zheru Chi,et al.  Improvement of Image Classification Using Wavelet Coefficients with Structured-Based Neural Network , 2008, Int. J. Neural Syst..

[32]  ASHIF PANAKKAT,et al.  Neural Network Models for Earthquake Magnitude Prediction Using Multiple seismicity Indicators , 2007, Int. J. Neural Syst..

[33]  Sampsa Laine,et al.  Visual approach to supervised variable selection by self-organizing map , 2005, Int. J. Neural Syst..

[34]  Ch. Venkateswarlu,et al.  Modeling and Optimization of a Pharmaceutical Formulation System Using Radial Basis Function Network , 2009, Int. J. Neural Syst..

[35]  W. Pedrycz.  An identification algorithm in fuzzy relational systems , 1984 .

[36]  Xiao-Shan Gao,et al.  Evolutionary programming based on non-uniform mutation , 2007, Appl. Math. Comput..

[37]  Sirish L. Shah,et al.  Choice of latent explanatory variables: a multiobjective optimization approach , 2000 .

[38]  Victor Yepes,et al.  Multiobjective Optimization of Concrete Frames by Simulated Annealing , 2008, Comput. Aided Civ. Infrastructure Eng..

[39]  Hojjat Adeli,et al.  Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings , 2007 .

[40]  Joel P. Conte,et al.  Damage Identification of a Composite Beam Using Finite Element Model Updating , 2008, Comput. Aided Civ. Infrastructure Eng..

[41]  Sankaran Mahadevan,et al.  Bayesian wavelet packet denoising for structural system identification , 2007 .

[42]  Valerie J Gillet,et al.  Multiobjective optimization in quantitative structure-activity relationships: deriving accurate and interpretable QSARs. , 2002, Journal of medicinal chemistry.

[43]  Witold Pedrycz,et al.  Implicit rule-based fuzzy-neural networks using the identification algorithm of GA hybrid scheme based on information granulation , 2002, Adv. Eng. Informatics.

[44]  Tipu Z. Aziz,et al.  Prediction of Parkinson's Disease tremor Onset Using a Radial Basis Function Neural Network Based on Particle Swarm Optimization , 2010, Int. J. Neural Syst..

[45]  Asim Karim,et al.  CONSCOM: An OO Construction Scheduling and Change Management System , 1999 .

[46]  Qingquan Wu,et al.  View invariant head recognition by Hybrid PCA based reconstruction , 2008 .

[47]  I. J. Leontaritis,et al.  Model selection and validation methods for non-linear systems , 1987 .

[48]  M Karplus,et al.  Evolutionary optimization in quantitative structure-activity relationship: an application of genetic neural networks. , 1996, Journal of medicinal chemistry.

[49]  Joel P. Conte,et al.  Modal Identification Study of Vincent Thomas Bridge Using Simulated Wind‐Induced Ambient Vibration Data , 2008, Comput. Aided Civ. Infrastructure Eng..

[50]  Dimitris K. Agrafiotis,et al.  A Novel Method for Building Regression Tree Models for QSAR Based on Artificial Ant Colony Systems , 2001, J. Chem. Inf. Comput. Sci..

[51]  Junfei Qiao,et al.  A Repair Algorithm for Radial Basis Function Neural Network and its Application to Chemical oxygen Demand Modeling , 2010, Int. J. Neural Syst..

[52]  Aini Hussain,et al.  An Intelligent Load Shedding Scheme Using Neural Networks and Neuro-Fuzzy , 2009, Int. J. Neural Syst..

[53]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[54]  Hojjat Adeli,et al.  Neuro‐genetic algorithm for non‐linear active control of structures , 2008 .

[55]  Sung Jin Cho,et al.  Genetic Algorithm Guided Selection: Variable Selection and Subset Selection , 2002, J. Chem. Inf. Comput. Sci..

[56]  Hojjat Adeli,et al.  Parallel backpropagation learning algorithms on CRAY Y-MP8/864 supercomputer , 1993, Neurocomputing.

[57]  Hojjat Adeli,et al.  A Concurrent Adaptive Conjugate Gradient Learning Algorithm On Mimd Shared-Memory Machines , 1993, Int. J. High Perform. Comput. Appl..

[58]  Hojjat Adeli,et al.  Wavelet‐Clustering‐Neural Network Model for Freeway Incident Detection , 2003 .

[59]  Hyo Seon Park,et al.  COUNTERPROPAGATION NEURAL NETWORKS IN STRUCTURAL ENGINEERING , 1995 .

[60]  Shu-Shen Liu,et al.  VSMP: A Novel Variable Selection and Modeling Method Based on the Prediction , 2003, J. Chem. Inf. Comput. Sci..

[61]  Hojjat Adeli,et al.  RADIAL BASIS FUNCTION NEURAL NETWORK FOR WORK ZONE CAPACITY AND QUEUE ESTIMATION , 2003 .

[62]  Drasko Furundzic,et al.  Application example of neural networks for time series analysis: : Rainfall-runoff modeling , 1998, Signal Process..

[63]  Euntai Kim,et al.  A Simple Identified Sugeno-Type Fuzzy Model via Double Clustering , 1998, Inf. Sci..

[64]  Juan Miguel Ortiz-de-Lazcano-Lobato,et al.  Dynamic Competitive Probabilistic Principal Components Analysis , 2009, Int. J. Neural Syst..

[65]  Anton J. Hopfinger,et al.  Application of Genetic Function Approximation to Quantitative Structure-Activity Relationships and Quantitative Structure-Property Relationships , 1994, J. Chem. Inf. Comput. Sci..

[66]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[67]  Hyo Seon Park,et al.  A neural dynamics model for structural optimization—Application to plastic design of structures , 1995 .

[68]  Hojjat Adeli,et al.  Distributed neural dynamics algorithms for optimization of large steel structures , 1997 .

[69]  Hojjat Adeli,et al.  Comparison of fuzzy-wavelet radial basis function neural network freeway incident detection model with California algorithm , 2002 .

[70]  Desire L. Massart,et al.  Variable selection for neural networks in multivariate calibration , 1998 .

[71]  J L Katz,et al.  2D QSAR modeling and preliminary database searching for dopamine transporter inhibitors using genetic algorithm variable selection of Molconn Z descriptors. , 2000, Journal of medicinal chemistry.

[72]  Hassan M. Elragal,et al.  Improving Neural Networks Prediction Accuracy Using Particle Swarm Optimization Combiner , 2009, Int. J. Neural Syst..

[73]  René V. Mayorga,et al.  A Radial Basis Function Network Approach for the Computation of Inverse Continuous Time Variant Functions , 2007, Int. J. Neural Syst..

[74]  Hojjat Adeli,et al.  A neural dynamics model for structural optimization—Theory , 1995 .

[75]  Haralambos Sarimveis,et al.  A Fast and Efficient Algorithm for Training Radial Basis Function Neural Networks Based on a Fuzzy Partition of the Input Space , 2002 .

[76]  Ju-Jang Lee,et al.  Heterogeneous local model networks for time series prediction , 2005, Appl. Math. Comput..

[77]  Hojjat Adeli,et al.  An Adaptive Conjugate Gradient Neural Network–Wavelet Model for Traffic Incident Detection , 2000 .

[78]  T. Næs,et al.  Principal component regression in NIR analysis: Viewpoints, background details and selection of components , 1988 .

[79]  Kazuyuki Murase,et al.  Faster Training Using Fusion of Activation Functions for Feed Forward Neural Networks , 2009, Int. J. Neural Syst..

[80]  Hojjat Adeli,et al.  Dynamic Wavelet Neural Network for Nonlinear Identification of Highrise Buildings , 2005 .

[81]  D. Agrafiotis,et al.  Feature selection for structure-activity correlation using binary particle swarms. , 2002, Journal of medicinal chemistry.

[82]  Mark J. L. Orr,et al.  Regularization in the Selection of Radial Basis Function Centers , 1995, Neural Computation.

[83]  Joaquín A. Pacheco,et al.  Analysis of new variable selection methods for discriminant analysis , 2006, Comput. Stat. Data Anal..

[84]  H. Adeli,et al.  Dynamic Fuzzy Wavelet Neural Network Model for Structural System Identification , 2006 .

[85]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[86]  Hojjat Adeli,et al.  FUZZY-WAVELET RBFNN MODEL FOR FREEWAY INCIDENT DETECTION , 2000 .

[87]  Sung-Kwun Oh,et al.  Identification of fuzzy systems by means of an auto-tuning algorithm and its application to nonlinear systems , 2000, Fuzzy Sets Syst..