Pruned search: A machine learning based meta-heuristic approach for constrained continuous optimization

Searching for solutions that optimize a continuous function can be difficult due to the infinite search space, and can be further complicated by the high dimensionality in the number of variables and complexity in the structure of constraints. Both deterministic and stochastic methods have been presented in the literature with a purpose of exploiting the search space and avoiding local optima as much as possible. In this research, we develop a machine learning framework aiming to `prune' the search effort of both types of optimization techniques by developing meta-heuristics, attempting to knowledgeably reordering the search space and reducing the search region. Numerical examples demonstrate that this approach can effectively find the global optimal solutions and significantly reduce the computational time for seven benchmark problems with variable dimensions of 100, 500 and 1000, compared to Genetic Algorithms.

[1]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[2]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[3]  Ganesh Ram Santhanam,et al.  Pavement Life-Cycle Sustainability Assessment and Interpretation Using a Novel Qualitative Decision Procedure , 2013, J. Comput. Civ. Eng..

[4]  Ankit Agrawal,et al.  Qualitative Decision Methods for Multi-Attribute Decision Making , 2015, ArXiv.

[5]  Vasant Honavar,et al.  Efficient Dominance Testing for Unconditional Preferences , 2010, KR.

[6]  G. T. Timmer,et al.  Stochastic global optimization methods part I: Clustering methods , 1987, Math. Program..

[7]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms for Constrained Parameter Optimization Problems , 1996, Evolutionary Computation.

[8]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[9]  Marco Sciandrone,et al.  Machine learning for global optimization , 2010, Computational Optimization and Applications.

[10]  Shie Mannor,et al.  Statistical Optimization in High Dimensions , 2012, Oper. Res..

[11]  Vasant Honavar,et al.  Representing and Reasoning with Qualitative Preferences for Compositional Systems , 2011, J. Artif. Intell. Res..

[12]  Arnold J. Stromberg,et al.  Number-theoretic Methods in Statistics , 1996 .

[13]  Mitsuo Gen,et al.  Genetic algorithms and engineering optimization , 1999 .

[14]  Wei-keng Liao,et al.  Search space preprocessing in solving complex optimization problems , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[15]  Alok Choudhary,et al.  A predictive machine learning approach for microstructure optimization and materials design , 2015, Scientific Reports.

[16]  Vasant Honavar,et al.  Dominance Testing via Model Checking , 2010, AAAI.

[17]  Yuan Wang,et al.  Some Applications of Number-Theoretic Methods in Statistics , 1994 .