Sample Efficient Graph-Based Optimization with Noisy Observations

We study sample complexity of optimizing ``hill-climbing friendly'' functions defined on a graph under noisy observations. We define a notion of convexity, and we show that a variant of best-arm identification can find a near-optimal solution after a small number of queries that is independent of the size of the graph. For functions that have local minima and are nearly convex, we show a sample complexity for the classical simulated annealing under noisy observations. We show effectiveness of the greedy algorithm with restarts and the simulated annealing on problems of graph-based nearest neighbor classification as well as a web advertising application.

[1]  Muni Sreenivas Pydi,et al.  Graph-Based Ascent Algorithms for Function Maximization , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[2]  R. Munos,et al.  Best Arm Identification in Multi-Armed Bandits , 2010, COLT.

[3]  Csaba Szepesvári,et al.  Online Optimization in X-Armed Bandits , 2008, NIPS.

[4]  Shie Mannor,et al.  Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems , 2006, J. Mach. Learn. Res..

[5]  Michal Valko,et al.  Simple regret for infinitely many armed bandits , 2015, ICML.

[6]  Rémi Munos,et al.  Spectral Bandits for Smooth Graph Functions , 2014, ICML.

[7]  Lydia E. Kavraki,et al.  Distributed computation of the knn graph for large high-dimensional point sets , 2007, J. Parallel Distributed Comput..

[8]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[9]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[10]  Clément Bouttier,et al.  Convergence Rate of a Simulated Annealing Algorithm with Noisy Observations , 2017, J. Mach. Learn. Res..

[11]  Jing Wang,et al.  Scalable k-NN graph construction for visual descriptors , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[13]  Reuven Y. Rubinstein,et al.  Optimization of computer simulation models with rare events , 1997 .

[14]  David Eppstein,et al.  On Nearest-Neighbor Graphs , 1992, ICALP.

[15]  Kai Li,et al.  Efficient k-nearest neighbor graph construction for generic similarity measures , 2011, WWW.

[16]  Rémi Munos,et al.  Black-box optimization of noisy functions with unknown smoothness , 2015, NIPS.

[17]  Yasin Abbasi-Yadkori,et al.  Fast Approximate Nearest-Neighbor Search with k-Nearest Neighbor Graph , 2011, IJCAI.

[18]  Sunil Arya,et al.  Algorithms for fast vector quantization , 1993, [Proceedings] DCC `93: Data Compression Conference.

[19]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[20]  Robert D. Nowak,et al.  Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting , 2014, 2014 48th Annual Conference on Information Sciences and Systems (CISS).

[21]  John N. Tsitsiklis,et al.  The Sample Complexity of Exploration in the Multi-Armed Bandit Problem , 2004, J. Mach. Learn. Res..

[22]  PlakuErion,et al.  Distributed computation of the knn graph for large high-dimensional point sets , 2007 .

[23]  Yousef Saad,et al.  Fast Approximate kNN Graph Construction for High Dimensional Data via Recursive Lanczos Bisection , 2009, J. Mach. Learn. Res..

[24]  M. R. Brito,et al.  Connectivity of the mutual k-nearest-neighbor graph in clustering and outlier detection , 1997 .

[25]  Csaba Szepesvári,et al.  Improved Algorithms for Linear Stochastic Bandits , 2011, NIPS.

[26]  Piyush Kumar,et al.  Fast construction of k-nearest neighbor graphs for point clouds , 2010, IEEE Transactions on Visualization and Computer Graphics.

[27]  Gary L. Miller,et al.  Separators for sphere-packings and nearest neighbor graphs , 1997, JACM.

[28]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..