Top-k Ranking Bayesian Optimization

This paper presents a novel approach to top-k ranking Bayesian optimization (top-k ranking BO) which is a practical and significant generalization of preferential BO to handle top-k ranking and tie/indifference observations. We first design a surrogate model that is not only capable of catering to the above observations, but is also supported by a classic random utility model. Another equally important contribution is the introduction of the first information-theoretic acquisition function in BO with preferential observation called multinomial predictive entropy search (MPES) which is flexible in handling these observations and optimized for all inputs of a query jointly. MPES possesses superior performance compared with existing acquisition functions that select the inputs of a query one at a time greedily. We empirically evaluate the performance of MPES using several synthetic benchmark functions, CIFAR-10 dataset, and SUSHI preference dataset.

[1]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection , 2018, J. Open Source Softw..

[2]  Bryan Kian Hsiang Low,et al.  Information-Based Multi-Fidelity Bayesian Optimization , 2017 .

[3]  Eyke Hüllermeier,et al.  Preference-based Online Learning with Dueling Bandits: A Survey , 2018, J. Mach. Learn. Res..

[4]  Zoubin Ghahramani,et al.  Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions , 2015, NIPS.

[5]  Neil D. Lawrence,et al.  Preferential Bayesian Optimization , 2017, ICML.

[6]  Wei Chu,et al.  Preference learning with Gaussian processes , 2005, ICML.

[7]  Kian Hsiang Low,et al.  Bayesian Optimization Meets Bayesian Optimal Stopping , 2019, ICML.

[8]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[9]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[10]  Kian Hsiang Low,et al.  Nonmyopic Gaussian Process Optimization with Macro-Actions , 2020, AISTATS.

[11]  Diego Granziol,et al.  Fast Information-theoretic Bayesian Optimisation , 2017, ICML.

[12]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[13]  Ian Dewancker,et al.  Sequential Preference-Based Optimization , 2018, ArXiv.

[14]  Philipp Hennig,et al.  Entropy Search for Information-Efficient Global Optimization , 2011, J. Mach. Learn. Res..

[15]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[16]  R. Plackett.  The Analysis of Permutations , 1975 .

[17]  Kian Hsiang Low,et al.  Bayesian Optimization with Binary Auxiliary Information , 2019, UAI.

[18]  Eric Walter,et al.  An informational approach to the global optimization of expensive-to-evaluate functions , 2006, J. Glob. Optim..

[19]  Víctor Cantillo,et al.  Thresholds and indifference in stated choice surveys , 2010 .

[20]  Zi Wang,et al.  Max-value Entropy Search for Efficient Bayesian Optimization , 2017, ICML.

[21]  D. McFadden.  MEASUREMENT OF URBAN TRAVEL DEMAND , 1974 .

[22]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[23]  Kian Hsiang Low,et al.  Decentralized High-Dimensional Bayesian Optimization with Factor Graphs , 2017, AAAI.

[24]  Kian Hsiang Low,et al.  Private Outsourced Bayesian Optimization , 2020, ICML.

[25]  Kian Hsiang Low,et al.  R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games , 2020, ICML.

[26]  Kian Hsiang Low,et al.  Distributed Batch Gaussian Process Optimization , 2017, ICML.

[27]  Ashish Khetan,et al.  Data-driven Rank Breaking for Efficient Rank Aggregation , 2016, J. Mach. Learn. Res..

[28]  Joel W. Burdick,et al.  Multi-dueling Bandits with Dependent Arms , 2017, UAI.

[29]  Valeria Vitelli,et al.  Probabilistic preference learning with the Mallows rank model , 2014, J. Mach. Learn. Res..

[30]  Florian Heiss,et al.  Discrete Choice Methods with Simulation , 2016 .

[31]  Toshihiro Kamishima,et al.  Nantonac collaborative filtering: recommendation based on order responses , 2003, KDD '03.

[32]  Matthew W. Hoffman,et al.  Predictive Entropy Search for Efficient Global Optimization of Black-box Functions , 2014, NIPS.

[33]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[34]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[35]  Kian Hsiang Low,et al.  Gaussian Process Planning with Lipschitz Continuous Reward Functions: Towards Unifying Bayesian Optimization, Active Learning, and Beyond , 2015, AAAI.

[36]  J. Marschak.  Binary Choice Constraints on Random Utility Indicators , 1959 .