Parameter Tuning for Self-optimizing Software at Scale

Efficiency of self-optimizing systems is heavily dependent on their optimization strategies, e.g., choosing exact or approximate solver. A choice of such a strategy, in turn, is influenced by numerous factors, such as re-optimization time, size of the problem, optimality constraints, etc. Exact solvers are domain-independent and can guarantee optimality but suffer from scaling, while approximate solvers offer a "good-enough" solution in exchange for a lack of generality and parameter-dependence. In this paper we discuss the trade-offs between exact and approximate optimizers for solving a quality-based software selection and hardware mapping problem from the scalability perspective. We show that even a simple heuristic can compete with thoroughly developed exact solvers under condition of an effective parameter tuning. Moreover, we discuss robustness of the obtained algorithm's configuration. Last but not least, we present a software product line for parameter tuning, which comprise the main features of this process and can serve as a platform for further research in the area of parameter tuning.

[1]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[2]  Kevin Leyton-Brown,et al.  Automated Configuration of Mixed Integer Programming Solvers , 2010, CPAIOR.

[3]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[4]  Mauro Birattari,et al.  Tuning Metaheuristics - A Machine Learning Perspective , 2009, Studies in Computational Intelligence.

[5]  Michel Camilleri An Algorithmic Approach to Parameter Selection in Machine Learning using Meta-Optimization Techniques , 2014 .

[6]  I. Sobol,et al.  A pseudo-random number generator for personal computers , 1999 .

[7]  Aaron Klein,et al.  Hyperparameter Optimization , 2017, Encyclopedia of Machine Learning and Data Mining.

[8]  Shekoufeh Kolahdouz Rahimi,et al.  Solving the Quality-based Software-Selection and Hardware-Mapping Problem with ACO , 2018, TTC@STAF.

[9]  Ardalan Vahidi,et al.  Optimal speed advisory for connected vehicles in arterial roads and the impact on mixed traffic , 2016 .

[10]  Aaron Klein,et al.  BOHB: Robust and Efficient Hyperparameter Optimization at Scale , 2018, ICML.

[11]  Sebastian Götz,et al.  BRISE: Energy-Efficient Benchmark Reduction , 2018, 2018 IEEE/ACM 6th International Workshop on Green And Sustainable Software (GREENS).

[12]  Yuan-Shun Dai,et al.  Interaction-based feature selection using Factorial Design , 2017, Neurocomputing.

[13]  Derek Bingham,et al.  Handbook of Design and Analysis of Experiments , 2015 .

[14]  Uwe Aßmann,et al.  Quality-based Software-Selection and Hardware-Mapping as Model Transformation Problem , 2018, TTC@STAF.

[15]  Minlan Yu,et al.  CherryPick: Adaptively Unearthing the Best Cloud Configurations for Big Data Analytics , 2017, NSDI.

[16]  Hui-Ling Huang,et al.  ESVM: Evolutionary support vector machine for automatic feature selection and classification of microarray data , 2007, Biosyst..

[17]  Gordon Fraser,et al.  On Parameter Tuning in Search Based Software Engineering , 2011, SSBSE.

[18]  Congbo Li,et al.  Multi-objective parameter optimization of CNC machining for low carbon manufacturing , 2015 .

[19]  Oscar Firschein,et al.  Readings in computer vision: issues, problems, principles, and paradigms , 1987 .

[20]  Celso C. Ribeiro,et al.  Scheduling in sports: An annotated bibliography , 2010, Comput. Oper. Res..

[21]  Uwe Aßmann,et al.  A Models@run.time Approach for Multi-objective Self-optimizing Software , 2014, ICAIS.

[22]  Dietmar Jannach,et al.  Efficient optimization of multiple recommendation quality factors according to individual user tendencies , 2017, Expert Syst. Appl..

[23]  Bogumił Kamiński,et al.  Choice of best possible metaheuristic algorithm for the travelling salesman problem with limited computational time: quality, uncertainty and speed , 2013 .

[24]  Uwe Aßmann,et al.  A JastAdd- and ILP-based Solution to the Software-Selection and Hardware-Mapping-Problem at the TTC 2018 , 2018, TTC@STAF.

[25]  Valentin Dalibard,et al.  BOAT: Building Auto-Tuners with Structured Bayesian Optimization , 2017, WWW.

[26]  Patrick Siarry,et al.  A survey on optimization metaheuristics , 2013, Inf. Sci..

[27]  H. Shakouri G.,et al.  Investigation on the choice of the initial temperature in the Simulated Annealing: A mushy state SA for TSP , 2009, 2009 17th Mediterranean Conference on Control and Automation.

[28]  D. Sculley,et al.  Google Vizier: A Service for Black-Box Optimization , 2017, KDD.

[29]  Walid Ben-Ameur,et al.  Computing the Initial Temperature of Simulated Annealing , 2004, Comput. Optim. Appl..

[30]  Richard M. Soland,et al.  Exact and approximate solutions to the multisource weber problem , 1972, Math. Program..