Computational Intelligence in Optimization

This chapter deals with the study of artificial neural networks (ANNs) and Heuristic Rules (HR) to solve optimization problems. The study of ANN as optimization tools for solving large scale problems was due to the fact that this technique has great potential for hardware VLSI implementation, in which it may be more efficient than traditional optimization techniques. However, the implementation of computational algorithm has shown that the proposed technique should have been efficient but slow as compared with traditional mathematical methods. In order to make it a fast method, we will show two ways to increase the speed of convergence of the computational algorithm. For analyzes and comparison, we solved three test cases. This paper considers recurrent ANN to solve linear and quadratic programming problems. These networks are based on the solution of a set of differential equations that are obtained from a transformation of an augmented Lagrange energy function. The proposed hybrid systems combining recurrent ANN and HR presented a reduced computational effort in relation to the one using only the recurrent ANN.

[1]  Robert B. Litterman,et al.  Global Portfolio Optimization , 1992 .

[2]  Shigeru Obayashi,et al.  Multidisciplinary design optimization of wing shape for a small jet aircraft using kriging model , 2006 .

[3]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[4]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[5]  R. Horst,et al.  Global Optimization: Deterministic Approaches , 1992 .

[6]  Jeffrey Horn,et al.  Handbook of evolutionary computation , 1997 .

[7]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[8]  Gregory Piatetsky-Shapiro,et al.  High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality , 2000 .

[9]  James P. Crutchfield,et al.  Evolving Globally Synchronized Cellular Automata , 1995, ICGA.

[10]  J J Hopfield,et al.  Learning algorithms and probability distributions in feed-forward and feed-back networks. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Eduardo Bayro-Corrochano,et al.  Geometric preprocessing, geometric feedforward neural networks and Clifford support vector machines for visual learning , 2005, Neurocomputing.

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Jinde Cao,et al.  A high performance neural network for solving nonlinear programming problems with hybrid constraints , 2001 .

[14]  Heinrich G. Jacob,et al.  Rechnergestützte Optimierung statischer und dynamischer Systeme , 1982 .

[15]  M. J. Norman,et al.  Monte Carlo Optimization, Simulation and Sensitivity of Queueing Networks , 1987 .

[16]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[17]  Terence C. Fogarty,et al.  Co-Evolving Co-Operative Populations of Rules in Learning Control Systems , 1994, Evolutionary Computing, AISB Workshop.

[18]  Kalyanmoy Deb,et al.  A Multi-Objective Optimization Procedure with Successive Approximate Models , .

[19]  Daniel Roggen,et al.  Multi-cellular Development: Is There Scalability and Robustness to Gain? , 2004, PPSN.

[20]  Deniz Yuret,et al.  Dynamic Hill Climbing: Overcoming the limitations of optimization techniques , 1993 .

[21]  Jeffrey L. Krichmar,et al.  Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines , 2001, Complex..

[22]  David Riaño,et al.  The scope of application of multi-agent systems in the process industry: three case studies , 2004, Expert Syst. Appl..

[23]  Carmen G. Moles,et al.  Parameter estimation in biochemical pathways: a comparison of global optimization methods. , 2003, Genome research.

[24]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[25]  Andy J. Keane,et al.  Computational Approaches for Aerospace Design: The Pursuit of Excellence , 2005 .

[26]  Leo G. Kroon,et al.  Routing trains through a railway station based on a node packing model , 2001, Eur. J. Oper. Res..

[27]  Jonathan L. Shapiro,et al.  Diversity Loss in General Estimation of Distribution Algorithms , 2006, PPSN.

[28]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[29]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[30]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[31]  Barbara Webb,et al.  Swarm Intelligence: From Natural to Artificial Systems , 2002, Connect. Sci..

[32]  S. Y. Kung,et al.  Parallel architectures for artificial neural nets , 1988, IEEE 1988 International Conference on Neural Networks.

[33]  Karl Sims,et al.  Evolving 3D Morphology and Behavior by Competition , 1994, Artificial Life.

[34]  R. Schaback.  Native Hilbert Spaces for Radial Basis Functions I , 1999 .

[35]  M. Cavaiuolo,et al.  A systolic neural network image processing architecture , 1992, CompEuro 1992 Proceedings Computer Systems and Software Engineering.

[36]  H. T. Kung.  Why systolic architectures? , 1982, Computer.

[37]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[38]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[39]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[40]  Michael A. Saunders,et al.  Inertia-Controlling Methods for General Quadratic Programming , 1991, SIAM Rev..

[41]  César Hervás-Martínez,et al.  Logistic regression using covariates obtained by product-unit neural network models , 2007, Pattern Recognit..

[42]  F. Girosi,et al.  Nonlinear prediction of chaotic time series using support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[43]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[44]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[45]  Christian Blum,et al.  Ant colony optimization: Introduction and recent trends , 2005 .

[46]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[47]  C. P. Pieters.  Reflections on the geno- and the phenotype , 2007 .

[48]  R.G. Girones,et al.  Systolic implementation of a pipelined on-line backpropagation , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[49]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[50]  Andreas Zell,et al.  Evaluating a hybrid encoding and three crossover operators on the constrained portfolio selection problem , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[51]  Hyun-Chul Kim,et al.  Support Vector Machine Ensemble with Bagging , 2002, SVM.

[52]  G. T. Timmer,et al.  Stochastic global optimization methods part I: Clustering methods , 1987, Math. Program..

[53]  Shigeru Obayashi,et al.  Multi-Objective Design Exploration for Aerodynamic Configurations , 2005 .

[54]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[55]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[56]  Marek Kisiel-Dorohinicki,et al.  The Application of Evolution Process in Multi-Agent World to the Prediction System , 1996 .

[57]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[58]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[59]  Andy J. Keane,et al.  A Derivative Based Surrogate Model for Approximating and Optimizing the Output of an Expensive Computer Simulation , 2004, J. Glob. Optim..

[60]  I-Tung Yang,et al.  Impact of budget uncertainty on project time-cost tradeoff , 2005, IEEE Transactions on Engineering Management.

[61]  Andy J. Keane,et al.  A Constraint Mapping Approach to the Structural Optimization of an Expensive Model using Surrogates , 2001 .

[62]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[63]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[64]  Mohamed Khalil Hani,et al.  Implementation of recurrent neural network algorithm for shortest path calculation in network routing , 2002, Proceedings International Symposium on Parallel Architectures, Algorithms and Networks. I-SPAN'02.

[65]  Slim Ben Saoud,et al.  Design and implementation of a neural command rule on a FPGA circuit , 2005, 2005 12th IEEE International Conference on Electronics, Circuits and Systems.

[66]  Derek L. G. Hill,et al.  Registration Methodology: Concepts and Algorithms , 2001 .

[67]  Jayadeva,et al.  Compact analogue neural network: a new paradigm for neural based combinatorial optimisation , 1999 .

[68]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[69]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[70]  Carlos Artemio Coello-Coello,et al.  Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art , 2002 .

[71]  Amine Bermak,et al.  Digital VLSI implementation of a multi-precision neural network classifier , 1999, ICONIP'99. ANZIIS'99 & ANNES'99 & ACNN'99. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378).

[72]  Lotfi A. Zadeh,et al.  Outline of a New Approach to the Analysis of Complex Systems and Decision Processes , 1973, IEEE Trans. Syst. Man Cybern..

[73]  Jun Wang,et al.  A general methodology for designing globally convergent optimization neural networks , 1998, IEEE Trans. Neural Networks.

[74]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[75]  Mahesan Niranjan,et al.  A systolic array implementation of a dynamic sequential neural network for pattern recognition , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[76]  Leszek Siwik,et al.  Agent-Based Co-Operative Co-Evolutionary Algorithm for Multi-Objective Optimization , 2006, ICAISC.

[77]  J. Strossmayera.  Dealings with Problem Hardness in Genetic Algorithms , 2009 .

[78]  G. Wahba,et al.  Multicategory Support Vector Machines , Theory , and Application to the Classification of Microarray Data and Satellite Radiance Data , 2004 .

[79]  Mancia Anguita,et al.  SCE Toolboxes for the Development of High-Level Parallel Applications , 2006, International Conference on Computational Science.

[80]  Elijah Polak,et al.  Optimization: Algorithms and Consistent Approximations , 1997 .

[81]  Holger Wendland.  Gaussian interpolation revisited , 2001 .

[82]  David Haussler,et al.  Exploiting Generative Models in Discriminative Classifiers , 1998, NIPS.

[83]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[84]  Tim D. Barfoot,et al.  Coevolving Communication and Cooperation for Lattice Formation Tasks , 2003, ECAL.

[85]  R. Axelrod.  More Effective Choice in the Prisoner's Dilemma , 1980 .

[86]  Jürgen Schmidhuber,et al.  Evolino: Hybrid Neuroevolution / Optimal Linear Search for Sequence Prediction , 2005, IJCAI 2005.

[87]  W. Verdini,et al.  Nonlinear time/cost tradeoff models in project management , 1995 .

[88]  Eckart Zitzler,et al.  Evolutionary algorithms for multiobjective optimization: methods and applications , 1999 .

[89]  John J. Hopfield,et al.  The effectiveness of analogue ‘neural network’ hardware , 1990 .

[90]  G. T. Timmer,et al.  Stochastic global optimization methods part II: Multi level methods , 1987, Math. Program..

[91]  K. M. Curtis,et al.  Efficient two-dimensional systolic array architecture for multilayer neural network , 1997 .

[92]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[93]  Bing J. Sheu,et al.  Neural information processing and VLSI , 1995 .

[94]  Kar-Ann Toh.  Global Optimization by Monotonic Transformation , 2002, Comput. Optim. Appl..

[95]  M. J. D. Powell,et al.  On the calculation of orthogonal vectors , 1968, Comput. J..

[96]  Mario Ventresca,et al.  Opposite Transfer Functions and Backpropagation Through Time , 2007, 2007 IEEE Symposium on Foundations of Computational Intelligence.

[97]  Jason Weston,et al.  Support vector machines for multi-class pattern recognition , 1999, ESANN.

[98]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[99]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[100]  Gunnar Rätsch,et al.  Predicting Time Series with Support Vector Machines , 1997, ICANN.

[101]  Simon King,et al.  Framewise phone classification using support vector machines , 2002, INTERSPEECH.

[102]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[103]  Hugh Glaser,et al.  A Genetic Approach to Understanding Cooperative Behaviour , 1996 .

[104]  S. Jones,et al.  A Performance Model for Multilayer Neural Networks in Linear Arrays , 1994, IEEE Trans. Parallel Distributed Syst..

[105]  Stephan Russenschuck,et al.  Integrated Design of Superconducting Magnets with the CERN Field Computation Program ROXIE , 2000 .

[106]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[107]  Michèle Sebag,et al.  Extending Population-Based Incremental Learning to Continuous Search Spaces , 1998, PPSN.

[108]  P. Gill,et al.  Quasi-Newton Methods for Unconstrained Optimization , 1972 .

[109]  Marco Dorigo,et al.  Evolving a Cooperative Transport Behavior for Two Simple Robots , 2003, Artificial Evolution.

[110]  Paul Cilliers,et al.  Boundaries , Hierarchies and Networks in Complex Systems , 2005 .

[111]  Frank Dellaert,et al.  Toward an evolvable model of development for autonomous agent synthesis , 1994 .

[112]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[113]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[114]  Chinchuan Chiu,et al.  An Artificial Neural Network Algorithm for Dynamic Programming , 1990, Int. J. Neural Syst..

[115]  Stephan Russenschuck,et al.  Using neural networks to speed up optimization algorithms , 2000 .

[116]  Alberto Suárez,et al.  Selection of Optimal Investment Portfolios with Cardinality Constraints , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[117]  S. Selcuk Erenguc,et al.  The resource constrained project scheduling problem with multiple crashable modes: An exact solution method , 2001 .

[118]  Gaurav S. Sukhatme,et al.  Collective construction with multiple robots , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[119]  N. Salvatore,et al.  A Surrogate Assisted Hooke-Jeeves Algorithm to Optimize the Control System of a PMSM Drive , 2006, 2006 IEEE International Symposium on Industrial Electronics.

[120]  D. Massicotte,et al.  A VLSI parallel architecture of a piecewise linear neural network for nonlinear channel equalization , 1999, IMTC/99. Proceedings of the 16th IEEE Instrumentation and Measurement Technology Conference (Cat. No.99CH36309).

[121]  C. Micchelli.  Interpolation of scattered data: Distance matrices and conditionally positive definite functions , 1986 .

[122]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[123]  Massimo A. Sivilotti,et al.  Real-time visual computations using analog CMOS processing arrays , 1987 .

[124]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[125]  Hussein A. Abbass,et al.  An evolutionary artificial neural networks approach for breast cancer diagnosis , 2002, Artif. Intell. Medicine.

[126]  J. R. Brown,et al.  Artificial neural network on a SIMD architecture , 1988, Proceedings., 2nd Symposium on the Frontiers of Massively Parallel Computation.

[127]  M. Zubair,et al.  Systolic implementation of neural networks , 1989, Proceedings 1989 IEEE International Conference on Computer Design: VLSI in Computers and Processors.

[128]  Michael A. Shanblatt,et al.  A two-phase optimization neural network , 1992, IEEE Trans. Neural Networks.

[129]  Matteo Fischetti,et al.  Modeling and Solving the Train Timetabling Problem , 2002, Oper. Res..

[130]  Marcus Randall,et al.  Anti-pheromone as a Tool for Better Exploration of Search Space , 2002, Ant Algorithms.

[131]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[132]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[133]  Carlos Eduardo Pedreira,et al.  Learning vector quantization with training data selection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[134]  M.M.A. Salama,et al.  Opposition-Based Differential Evolution , 2008, IEEE Transactions on Evolutionary Computation.

[135]  L. Penrose,et al.  Self-Reproducing Machines , 1959 .

[136]  Mohamed S. Kamel,et al.  Opposition-Based Q(lambda) Algorithm. , 2006, ISNN 2006.

[137]  Mario Vanhoucke,et al.  The discrete time/cost trade-off problem: extensions and heuristic procedures , 2007 .

[138]  Christopher R. Houck,et al.  A Genetic Algorithm for Function Optimization: A Matlab Implementation , 2001 .

[139]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[140]  Yusin Lee,et al.  Modeling and solving the train pathing problem , 2008 .

[141]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[142]  Rafal Drezewski,et al.  A Model of Co-evolution in Multi-agent System , 2003, CEEMAS.

[143]  K. A. Robinson.  Dictionary of Eye Terminology , 1997 .

[144]  F. Pazienti.  A systolic array for neural network implementation , 1991, [1991 Proceedings] 6th Mediterranean Electrotechnical Conference.

[145]  Marco Laumanns,et al.  A Spatial Predator-Prey Approach to Multi-objective Optimization: A Preliminary Study , 1998, PPSN.

[146]  Shenghuo Zhu,et al.  Improving medical/biological data classification performance by wavelet preprocessing , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[147]  P.K. Meher,et al.  Systolic array realization of a neural network-based face recognition system , 2008, 2008 3rd IEEE Conference on Industrial Electronics and Applications.

[148]  Hugh Glaser,et al.  The Prisoners' Dilemma Revisited , 1996 .

[149]  Eduardo Bayro-Corrochano,et al.  Recurrent Clifford Support Machines , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[150]  Richard K. Belew,et al.  Evolving robot morphology and control , 2006, Artificial Life and Robotics.

[151]  Sanjay Srivastava,et al.  Multi-resource-constrained discrete time-cost tradeoff with MOGA based hybrid method , 2007, 2007 IEEE Congress on Evolutionary Computation.

[152]  Jenq-Neng Hwang,et al.  A unifying algorithm/architecture for artificial neural networks , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[153]  Sou-Sen Leu,et al.  A GA-based fuzzy optimal model for construction time-cost trade-off , 2001 .

[154]  Faustino J. Gomez,et al.  Recurrent Support Vector Machines , 2005 .

[155]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[156]  M. A. Bayoumi,et al.  A reconfigurable 'ANN' architecture , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[157]  A. Mackay.  On complexity , 2001 .

[158]  Colin Campbell,et al.  Bayes Point Machines , 2001, J. Mach. Learn. Res..

[159]  David S. Broomhead,et al.  A systolic array for nonlinear adaptive filtering and pattern recognition , 1991, J. VLSI Signal Process..

[160]  David Corne,et al.  The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[161]  Elias S. Manolakos,et al.  A VLSI array architecture for the on-line training of recurrent neural networks , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[162]  Deeparnab Chakrabarty,et al.  Knapsack Problems , 2008 .

[163]  Maciej Komosinski,et al.  Framsticks: Towards a Simulation of a Nature-Like World, Creatures and Evolution , 1999, ECAL.

[164]  Hamid R. Tizhoosh,et al.  Opposition-Based Reinforcement Learning , 2006, J. Adv. Comput. Intell. Intell. Informatics.

[165]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[166]  David E. Goldberg,et al.  What Makes a Problem Hard for a Classifier System , 1992 .

[167]  Rolf Pfeifer,et al.  Understanding intelligence , 2020, Inequality by Design.

[168]  Chinchuan Chiu,et al.  Energy function analysis of dynamic programming neural networks , 1991, IEEE Trans. Neural Networks.

[169]  D Psaltis,et al.  Optical implementation of the Hopfield model. , 1985, Applied optics.

[170]  Mauricio G. C. Resende,et al.  An implementation of Karmarkar's algorithm for linear programming , 1989, Math. Program..

[171]  Hamid R. Tizhoosh,et al.  Reinforcement Learning Based on Actions and Opposite Actions , 2005 .

[172]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[173]  W. Madych,et al.  Multivariate interpolation and condi-tionally positive definite functions , 1988 .

[174]  Leszek Siwik,et al.  Agent-Based Co-Evolutionary Techniques for Solving Multi-Objective Optimization Problems , 2008 .

[175]  J. Suykens,et al.  Recurrent least squares support vector machines , 2000 .

[176]  J. Shapcott.  Index Tracking : Genetic Algorithms for Investment Portfolio Selection , 2002 .

[177]  Michael A. Shanblatt,et al.  Linear and quadratic programming neural network analysis , 1992, IEEE Trans. Neural Networks.

[178]  M. Nirmala Devi,et al.  FPGA Realization of Activation Function for Artificial Neural Networks , 2008, 2008 Eighth International Conference on Intelligent Systems Design and Applications.

[179]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[180]  Goutam Chakraborty,et al.  Neural-Network-Based Robust Linearization and Compensation Technique for Sensors Under Nonlinear Environmental Influences , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[181]  Joshua D. Knowles,et al.  Multiobjective Optimization on a Budget of 250 Evaluations , 2005, EMO.

[182]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[183]  Jayadeva,et al.  Learning To Optimize VLSI Design Problems , 2006, 2006 Annual IEEE India Conference.

[184]  Jaume Bacardit,et al.  Smart crossover operator with multiple parents for a Pittsburgh learning classifier system , 2006, GECCO '06.

[185]  Mario Ventresca,et al.  Simulated Annealing with Opposite Neighbors , 2007, 2007 IEEE Symposium on Foundations of Computational Intelligence.

[186]  Leszek Siwik,et al.  The Application of Agent-Based Co-Evolutionary System with Predator-Prey Interactions to Solving Multi-Objective Optimization Problems , 2007, 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making.

[187]  K. W. Przytula,et al.  Mapping of neural networks onto programmable parallel machines , 1990, IEEE International Symposium on Circuits and Systems.

[188]  Béat Hirsbrunner,et al.  Implicit Cooperation and Antagonism in Multi-Agent Systems , 1996 .

[189]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[190]  E. H. Mandami.  Application of Fuzzy Logic to Approximate Reasoning using Linguistic Synthesis , 1977 .

[191]  Nam Ling,et al.  Systolic architectures for artificial neural nets , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[192]  Hong Zhang,et al.  Blind bulldozing: multiple robot nest construction , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[193]  Melanie Mitchell,et al.  Complexity - A Guided Tour , 2009 .

[194]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[195]  K Wagner,et al.  Multilayer optical learning networks. , 1987, Applied optics.

[196]  Ulrich Ramacher,et al.  Architecture and VLSI design of a VLSI neural signal processor , 1993, 1993 IEEE International Symposium on Circuits and Systems.

[197]  Andrew B. Kahng,et al.  A new adaptive multi-start technique for combinatorial global optimizations , 1994, Oper. Res. Lett..

[198]  J. J. Hopfield,et al.  ‘Unlearning’ has a stabilizing effect in collective memories , 1983, Nature.

[199]  Prabuddha De,et al.  Complexity of the Discrete Time-Cost Tradeoff Problem for Project Networks , 1997, Oper. Res..

[200]  Mario Ventresca,et al.  Improving the Convergence of Backpropagation by Opposite Transfer Functions , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[201]  Diego Federici,et al.  Evolution and Development of a Multicellular Organism: Scalability, Resilience, and Neutral Complexification , 2006, Artificial Life.

[202]  John H. Holland.  Genetic Algorithms and Classifier Systems: Foundations and Future Directions , 1987, ICGA.

[203]  J. Bendor,et al.  Effective Choice in the Prisoner ' s Dilemma , 2007 .

[204]  Mario Vanhoucke,et al.  New computational results for the discrete time/cost trade-off problem with time-switch constraints , 2005, Eur. J. Oper. Res..

[205]  J.M. Moreno,et al.  An analog systolic neural processing architecture , 1994, IEEE Micro.

[206]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[207]  Young-Jin Jang,et al.  A programmable digital neuro-processor design with dynamically reconfigurable pipeline/parallel architecture , 1998, Proceedings 1998 International Conference on Parallel and Distributed Systems (Cat. No.98TB100250).

[208]  Myeong-Wuk Jang,et al.  Cooperation in Multi-agent Systems , 1995 .

[209]  Majid Sarrafzadeh,et al.  Congestion minimization during placement , 2000, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[210]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[211]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[212]  Jekanthan Thangavelautham,et al.  A Coarse-Coding Framework for a Gene-Regulatory-Based Artificial Neural Tissue , 2005, ECAL.

[213]  C. P. Pieters.  Effective Adaptive Plans , 2006 .

[214]  Mario Ventresca,et al.  A diversity maintaining population-based incremental learning algorithm , 2008, Inf. Sci..

[215]  Thomas Hofmann,et al.  Hidden Markov Support Vector Machines , 2003, ICML.

[216]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[217]  Isabelle Guyon,et al.  Comparison of classifier methods: a case study in handwritten digit recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[218]  Prabuddha De,et al.  The discrete time-cost tradeoff problem revisited , 1995 .

[219]  Nicholas J. Radcliffe,et al.  Equivalence Class Analysis of Genetic Algorithms , 1991, Complex Syst..

[220]  C. Lemieux.  Monte Carlo and Quasi-Monte Carlo Sampling , 2009 .

[221]  Hugh Glaser,et al.  Parallel Implementation of a Genetic-Programming Based Tool for Symbolic Regression , 1998, Inf. Process. Lett..

[222]  Antonio F. Gómez-Skarmeta,et al.  Approximative fuzzy rules approaches for classification with hybrid-GA techniques , 2001, Inf. Sci..

[223]  David Hestenes,et al.  New algebraic tools for classical geometry , 2001 .

[224]  Gündüz Ulusoy,et al.  A survey on the resource-constrained project scheduling problem , 1995 .

[225]  Yoh-Han Pao,et al.  Combinatorial optimization with use of guided evolutionary simulated annealing , 1995, IEEE Trans. Neural Networks.

[226]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[227]  L. Dagum,et al.  OpenMP: an industry standard API for shared-memory programming , 1998 .

[228]  Ronald J. Williams,et al.  Experimental Analysis of the Real-time Recurrent Learning Algorithm , 1989 .

[229]  Carlos A. Coello Coello,et al.  An updated survey of GA-based multiobjective optimization techniques , 2000, CSUR.

[230]  K. M. Curtis,et al.  Two-ring systolic array network for artificial neural networks , 1999 .

[231]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[232]  Leszek Siwik,et al.  Co-Evolutionary Multi-Agent System for Portfolio Optimization , 2008, Natural Computing in Computational Finance.

[233]  Jekanthan Thangavelautham,et al.  A Neuroevolutionary Approach to Emergent Task Decomposition , 2004, PPSN.

[234]  Oscal T.-C. Chen,et al.  Neural-based analog trainable vector quantizer and digital systolic processors , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[235]  R. E. Abdel-Aal,et al.  GMDH-based feature ranking and selection for improved classification of medical data , 2005, J. Biomed. Informatics.

[236]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[237]  Kee-Eung Kim,et al.  Statistical Machine Learning for Large-Scale Optimization , 2000 .

[238]  J.F. Myoupo,et al.  A single-layer systolic architecture for backpropagation learning , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[239]  Chung-Wei Feng,et al.  Using genetic algorithms to solve construction time-cost trade-off problems , 1997 .

[240]  Michael Emmerich,et al.  Metamodel Assisted Multiobjective Optimisation Strategies and their Application in Airfoil Design , 2004 .

[241]  George G. Robertson,et al.  Parallel Implementation of Genetic Algorithms in a Classifier Rystem , 1987, ICGA.

[242]  P.K. Meher,et al.  An embedded face recognition system on A VLSI array architecture and its FPGA implementation , 2008, 2008 34th Annual Conference of IEEE Industrial Electronics.

[243]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[244]  Rainer Kolisch,et al.  Experimental investigation of heuristics for resource-constrained project scheduling: An update , 2006, Eur. J. Oper. Res..

[245]  C. P. Pieters,et al.  Complex Systems and Patterns , 2008 .

[246]  Andries Petrus Engelbrecht,et al.  Differential evolution methods for unsupervised image classification , 2005, 2005 IEEE Congress on Evolutionary Computation.

[247]  Antoine Bordes,et al.  The Huller: A Simple and Efficient Online SVM , 2005, ECML.

[248]  Michael Peter Kennedy,et al.  Unifying the Tank and Hopfield linear programming circuit and the canonical nonlinear programming circuit of Chua and Lin , 1987 .

[249]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[250]  Maja J. Matarić,et al.  Perceptuo-Motor Primitives in Imitation , 1998 .

[251]  Shahryar Rahnamayan,et al.  Opposition-Based Differential Evolution Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[252]  David E. Goldberg,et al.  A niched Pareto genetic algorithm for multiobjective optimization , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[253]  S. Y. Kung.  Tutorial: digital neurocomputing for signal/image processing , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.

[254]  Jean-Louis Deneubourg,et al.  From local actions to global tasks: stigmergy and collective robotics , 2000 .

[255]  Andy J. Keane,et al.  Multi-Objective Optimization Using Surrogates , 2010 .

[256]  M. Forina,et al.  Multivariate calibration. , 2007, Journal of chromatography. A.

[257]  Vijayan K. Asari,et al.  A high speed flat CORDIC based neuron with multi-level activation function for robust pattern recognition , 2000, Proceedings Fifth IEEE International Workshop on Computer Architectures for Machine Perception.

[258]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[259]  Kemal Polat,et al.  Automatic detection of heart disease using an artificial immune recognition system (AIRS) with fuzzy resource allocation mechanism and k , 2007, Expert Syst. Appl..

[260]  Zhi-You Wu,et al.  A Novel monotonization Transformation for Some Classes of Global Optimization Problems , 2006, Asia Pac. J. Oper. Res..

[261]  Robert Schaback,et al.  Error estimates and condition numbers for radial basis function interpolation , 1995, Adv. Comput. Math..

[262]  Jari Toivanen,et al.  EVOLUTIONARY METHODS FOR DESIGN , OPTIMISATION AND , 2007 .

[263]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[264]  Xavier Gandibleux,et al.  A survey and annotated bibliography of multiobjective combinatorial optimization , 2000, OR Spectr..

[265]  P. Maître.  PLAYING FAIR GAME THEORY AND THE SOCIAL CONTRACT , 1994 .

[266]  Kenneth O. Stanley,et al.  A Case Study on the Critical Role of Geometric Regularity in Machine Learning , 2008, AAAI.

[267]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[268]  Holger Wendland,et al.  Approximate Interpolation with Applications to Selecting Smoothing Parameters , 2005, Numerische Mathematik.

[269]  Jihong Liu,et al.  A Survey of FPGA-Based Hardware Implementation of ANNs , 2005, 2005 International Conference on Neural Networks and Brain.

[270]  Andrew W. Moore,et al.  Learning evaluation functions for global optimization , 1998 .

[271]  Chris Melhuish,et al.  Algorithms for Building Annular Structures with Minimalist Robots Inspired by Brood Sorting in Ant Colonies , 2004, Auton. Robots.

[272]  Stamatis Vassiliadis,et al.  Sigmoid Generators for Neural Computing Using Piecewise Approximations , 1996, IEEE Trans. Computers.

[273]  S. Thomas Ng,et al.  Stochastic Time–Cost Optimization Model Incorporating Fuzzy Sets Theory and Nonreplaceable Front , 2005 .

[274]  Andrew W. Moore,et al.  Learning Evaluation Functions for Global Optimization and Boolean Satisfiability , 1998, AAAI/IAAI.

[275]  L. Margulis.  Symbiotic Planet: A New Look At Evolution , 1998 .

[276]  Insley B. Pyne,et al.  Linear programming on an electronic analogue computer , 1956, Transactions of the American Institute of Electrical Engineers, Part I: Communication and Electronics.

[277]  G. Weinberg.  An Introduction to General Systems Thinking , 1975 .

[278]  Erik Demeulemeester,et al.  Project scheduling : a research handbook , 2002 .

[279]  Alexander H. G. Rinnooy Kan,et al.  Concurrent stochastic methods for global optimization , 1990, Math. Program..

[280]  Lashon B. Booker,et al.  Improving the Performance of Genetic Algorithms in Classifier Systems , 1985, ICGA.

[281]  R. Pfeifer,et al.  Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny , 2001 .

[282]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[283]  Sanjay Srivastava,et al.  MOGA-based time-cost tradeoffs: Responsiveness for project uncertainties , 2007, 2007 IEEE Congress on Evolutionary Computation.

[284]  Frank Kursawe,et al.  A Variant of Evolution Strategies for Vector Optimization , 1990, PPSN.

[285]  Reiko Tanese,et al.  Distributed Genetic Algorithms , 1989, ICGA.

[286]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[287]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[288]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[289]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[290]  A. Lindenmayer.  Mathematical models for cellular interactions in development. I. Filaments with one-sided inputs. , 1968, Journal of theoretical biology.

[291]  Michael E. Bratman,et al.  Shared Cooperative Activity , 1991 .

[292]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[293]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[294]  C. P. Pieters.  A Pattern-Oriented Approach to Health; Using PAC in a Discourse of Health , 2009 .

[295]  Gérard Dreyfus,et al.  Single-layer learning revisited: a stepwise procedure for building and training a neural network , 1989, NATO Neurocomputing.

[296]  R. Detrano,et al.  International application of a new probability algorithm for the diagnosis of coronary artery disease. , 1989, The American journal of cardiology.

[297]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[298]  Martin Nilsson,et al.  Cooperative multi-robot box-pushing , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[299]  Kristin P. Bennett,et al.  A Pattern Search Method for Model Selection of Support Vector Regression , 2002, SDM.

[300]  Bruce George Linster.  Essays on cooperation and competition. , 1990 .

[301]  Gloria E. Phillips-Wren,et al.  Innovations in multi-agent systems , 2007, J. Netw. Comput. Appl..

[302]  W. R. Madych,et al.  Miscellaneous error bounds for multiquadric and related interpolators , 1992 .

[303]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[304]  R. Oeuvray.  Trust-region methods based on radial basis functions with application to biomedical imaging , 2005 .

[305]  Michael Mikolajczak,et al.  Designing And Building Parallel Programs: Concepts And Tools For Parallel Software Engineering , 1997, IEEE Concurrency.

[306]  Ali M. S. Zalzala,et al.  Recent developments in evolutionary computation for manufacturing optimization: problems, solutions, and comparisons , 2000, IEEE Trans. Evol. Comput..

[307]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[308]  Jekanthan Thangavelautham,et al.  Evolving a Scalable Multirobot Controller Using an Artificial Neural Tissue Paradigm , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[309]  Risto Miikkulainen,et al.  Continual Coevolution Through Complexification , 2002, GECCO.

[310]  Amir Azaron,et al.  A genetic algorithm approach for the time-cost trade-off in PERT networks , 2005, Appl. Math. Comput..

[311]  David B. Fogel,et al.  Evolving an expert checkers playing program without using human expertise , 2001, IEEE Trans. Evol. Comput..

[312]  Sooyong Park,et al.  Designing multi-agent systems: a framework and application , 2005, Expert Syst. Appl..

[313]  Jagdish Chandra Patra,et al.  Field Programmable Gate Array Implementation of a Neural Network-based Intelligent Sensor System , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[314]  John H. Holland,et al.  Properties of the bucket brigade algorithm , 1985 .

[315]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[316]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[317]  Jouni Sampo,et al.  Weighted Similarity Classifier Using Differential Evolution and Genetic Algorithm in Weight Optimization , 2004, J. Adv. Comput. Intell. Intell. Informatics.

[318]  Michael L. Littman,et al.  Efficient Reinforcement Learning with Relocatable Action Models , 2007, AAAI.

[319]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[320]  A. Balasubramaniam,et al.  A learning strategy for multilayer neural network using discretized Sigmoidal function , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[321]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[322]  Peter J. Fleming,et al.  Multiobjective optimization and multiple constraint handling with evolutionary algorithms. II. Application example , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[323]  S. Russenschuck,et al.  An improved method using radial basis function neural networks to speed up optimization algorithms , 2002 .

[324]  Zhaoyu Wang,et al.  Global versus Local Optimization in Redundancy Resolution of Robotic Manipulators , 1988, Int. J. Robotics Res..

[325]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[326]  Sou-Sen Leu,et al.  GA-BASED MULTICRITERIA OPTIMAL MODEL FOR CONSTRUCTION SCHEDULING , 1999 .

[327]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[328]  Liang Shi,et al.  Multiobjective GA optimization using reduced models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).