Evolutive Approaches for Variable Selection Using a Non-parametric Noise Estimator

The design of a model to approximate a function relies significantly on the data used in the training stage. The problem of selecting an adequate set of variables should be treated carefully due to its importance. If the number of variables is high, the number of samples needed to design the model becomes too large and the interpretability of the model is lost. This chapter presents several methodologies to perform variable selection in a local or a globalmanner using a non-parametric noise estimator to determine the quality of a subset of variables. Several methods that apply parallel paradigms in different architecures are compared from the optimization and efficiency point of view since the problem is computationally expensive.

[1]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[3]  Dirk Thierens,et al.  Mixing in Genetic Algorithms , 1993, ICGA.

[4]  A.H. Mantawy,et al.  A new tabu search algorithm for the long-term hydro scheduling problem , 2002, LESCOPE'02. 2002 Large Engineering Systems Conference on Power Engineering. Conference Proceedings.

[5]  Tomoyuki Hiroyasu,et al.  Distributed genetic algorithms with randomized migration rate , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[6]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[7]  Fred W. Glover Parametric tabu-search for mixed integer programs , 2006, Comput. Oper. Res..

[8]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[9]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[10]  Michel Verleysen,et al.  Using the Delta Test for Variable Selection , 2008, ESANN.

[11]  K. Al-Sultan,et al.  A tabu search Hooke and Jeeves algorithm for unconstrained optimization , 1997 .

[12]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[13]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[14]  Michel Barlaud,et al.  Fast k nearest neighbor search using GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[15]  Ginés Rubio,et al.  Design of specific-to-problem kernels and use of kernel weighted K-nearest neighbours for time series modelling , 2010, Neurocomputing.

[16]  Héctor Pomares,et al.  Variable Selection in a GPU Cluster Using Delta Test , 2011, IWANN.

[17]  Erick Cantú-Paz,et al.  Markov chain models of parallel genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[18]  Ignacio Rojas,et al.  Minimising the delta test for variable selection in regression problems , 2008, Int. J. High Perform. Syst. Archit..

[19]  Héctor Pomares,et al.  Improving Clustering Technique for Functional Approximation Problem Using Fuzzy Logic: ICFA Algorithm , 2005, IWANN.

[20]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[21]  Abdul Sattar,et al.  AI 2006: Advances in Artificial Intelligence, 19th Australian Joint Conference on Artificial Intelligence, Hobart, Australia, December 4-8, 2006, Proceedings , 2006, Australian Conference on Artificial Intelligence.

[22]  Richard J. Enbody,et al.  Further Research on Feature Selection and Classification Using Genetic Algorithms , 1993, ICGA.

[23]  Kalyanmoy Deb,et al.  Controlled Elitist Non-dominated Sorting Genetic Algorithms for Better Convergence , 2001, EMO.

[24]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[25]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[26]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[27]  Héctor Pomares,et al.  Boosting the Performance of a Multiobjective Algorithm to Design RBFNNs Through Parallelization , 2007, ICANNGA.

[28]  Kalyanmoy Deb,et al.  Analysis of Selection Algorithms: A Markov Chain Approach , 1996, Evolutionary Computation.

[29]  Melanie Mitchell,et al.  Genetic algorithms and artificial life , 1994 .

[30]  Francisco Herrera,et al.  Gradual distributed real-coded genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[31]  Peigen Li,et al.  A tabu search algorithm with a new neighborhood structure for the job shop scheduling problem , 2007, Comput. Oper. Res..

[32]  Terence C. Fogarty,et al.  Varying the Probability of Mutation in the Genetic Algorithm , 1989, ICGA.

[33]  L. Wang,et al.  VHDL-AMS based genetic optimization of a fuzzy logic controller for automotive active suspension systems , 2005, BMAS 2005. Proceedings of the 2005 IEEE International Behavioral Modeling and Simulation Workshop, 2005..

[34]  Hisao Ishibuchi,et al.  Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling , 2003, IEEE Trans. Evol. Comput..

[35]  Francisco Luna,et al.  Advances in parallel heterogeneous genetic algorithms for continuous optimization , 2004 .

[36]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[37]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[38]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[39]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[40]  Carsten Peterson,et al.  Finding the Embedding Dimension and Variable Dependencies in Time Series , 1994, Neural Computation.

[41]  Antonia J. Jones,et al.  New tools in non-linear modelling and prediction , 2004, Comput. Manag. Sci..

[42]  Stephan Scheuerer,et al.  A tabu search heuristic for the truck and trailer routing problem , 2006, Comput. Oper. Res..

[43]  Bart Kosko,et al.  Fuzzy Systems as Universal Approximators , 1994, IEEE Trans. Computers.

[44]  Héctor Pomares,et al.  Using fuzzy logic to improve a clustering technique for function approximation , 2007, Neurocomputing.

[45]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[46]  Colin R. Reeves,et al.  Using Genetic Algorithms with Small Populations , 1993, ICGA.

[47]  Héctor Pomares,et al.  GPU Cluster with MATLAB , 2011 .

[48]  Fred W. Glover,et al.  Using tabu search to solve the Steiner tree-star problem in telecommunications network design , 1996, Telecommun. Syst..

[49]  Mauro Dell'Amico,et al.  Applying tabu search to the job-shop scheduling problem , 1993, Ann. Oper. Res..

[50]  Héctor Pomares,et al.  Output value-based initialization for radial basis function neural networks , 2007, Neural Processing Letters.

[51]  Héctor Pomares,et al.  Parallel Multi-objective Memetic RBFNNs Design and Feature Selection for Function Approximation Problems , 2007, IWANN.

[52]  Jan Wessnitzer,et al.  A Model of Non-elemental Associative Learning in the Mushroom Body Neuropil of the Insect Brain , 2007, ICANNGA.

[53]  Byung Ro Moon,et al.  Local search-embedded genetic algorithms for feature selection , 2002, Object recognition supported by user interaction for service robots.

[54]  Héctor Pomares,et al.  The TaSe-NF model for function approximation problems: Approaching local and global modelling , 2011, Fuzzy Sets Syst..

[55]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[56]  Hans-Paul Schwefel,et al.  Advances in Computational Intelligence , 2003, Natural Computing Series.

[57]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[58]  Masoud Nikravesh,et al.  Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing) , 2006 .

[59]  Kenneth A. De Jong,et al.  An Analysis of the Interacting Roles of Population Size and Crossover in Genetic Algorithms , 1990, PPSN.

[60]  Masao Fukushima,et al.  Tabu Search directed by direct search methods for nonlinear global optimization , 2006, Eur. J. Oper. Res..

[61]  Alessandro Verri,et al.  Pattern Recognition with Support Vector Machines , 2002, Lecture Notes in Computer Science.

[62]  Héctor Pomares,et al.  Effective Input Variable Selection for Function Approximation , 2006, ICANN.

[63]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[64]  Sikun Li,et al.  Extracting Minimum Unsatisfiable Cores with a Greedy Genetic Algorithm , 2006, Australian Conference on Artificial Intelligence.

[65]  José Brandão,et al.  A tabu search algorithm for the open vehicle routing problem , 2004, Eur. J. Oper. Res..

[66]  Héctor Pomares,et al.  Improving the Performance of Multi-objective Genetic Algorithm for Function Approximation Through Parallel Islands Specialisation , 2006, Australian Conference on Artificial Intelligence.

[67]  Marc Toussaint,et al.  Extracting Motion Primitives from Natural Handwriting Data , 2006, ICANN.

[68]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[69]  Héctor Pomares,et al.  Parallel multiobjective memetic RBFNNs design and feature selection for function approximation problems , 2009, Neurocomputing.

[70]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.