Adaptive Sequence Submodularity

In many machine learning applications, one needs to interactively select a sequence of items (e.g., recommending movies based on a user's feedback) or make sequential decisions in a certain order (e.g., guiding an agent through a series of states). Not only do sequences already pose a dauntingly large search space, but we must also take into account past observations, as well as the uncertainty of future outcomes. Without further structure, finding an optimal sequence is notoriously challenging, if not completely intractable. In this paper, we view the problem of adaptive and sequential decision making through the lens of submodularity and propose an adaptive greedy policy with strong theoretical guarantees. Additionally, to demonstrate the practical utility of our results, we run experiments on Amazon product recommendation and Wikipedia link prediction tasks.

[1]  Andreas Krause,et al.  Near-optimal Batch Mode Active Learning and Adaptive Submodular Optimization , 2013, ICML.

[2]  Eric Balkanski,et al.  An Exponential Speedup in Parallel Running Time for Submodular Maximization without Loss in Approximation , 2018, SODA.

[3]  Andreas Krause,et al.  Distributed Submodular Maximization: Identifying Representative Elements in Massive Data , 2013, NIPS.

[4]  Jeff A. Bilmes,et al.  Submodularity for Data Selection in Statistical Machine Translation , 2014 .

[5]  Éva Tardos,et al.  Maximizing the Spread of Influence through a Social Network , 2015, Theory Comput..

[6]  Edwin K. P. Chong,et al.  String Submodular Functions With Curvature Constraints , 2013, IEEE Transactions on Automatic Control.

[7]  Alexandros G. Dimakis,et al.  Scalable Greedy Feature Selection via Weak Submodularity , 2017, AISTATS.

[8]  Andreas Krause,et al.  Near-optimal Nonmyopic Value of Information in Graphical Models , 2005, UAI.

[9]  Andreas Krause,et al.  Submodularity on Hypergraphs: From Sets to Sequences , 2018, AISTATS.

[10]  H. B. McMahan,et al.  Robust Submodular Observation Selection , 2008 .

[11]  Andreas Krause,et al.  Near-Optimally Teaching the Crowd to Classify , 2014, ICML.

[12]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[13]  Silvio Lattanzi,et al.  Submodular Streaming in All its Glory: Tight Approximation, Minimum Memory and Low Adaptive Complexity , 2019, ICML.

[14]  Morteza Zadimoghaddam,et al.  Scalable Deletion-Robust Submodular Maximization: Data Summarization with Privacy and Fairness Constraints , 2018, ICML.

[15]  Morteza Zadimoghaddam,et al.  Data Summarization at Scale: A Two-Stage Submodular Approach , 2018, ICML.

[16]  Volkan Cevher,et al.  Robust Submodular Maximization: A Non-Uniform Partitioning Approach , 2017, ICML.

[17]  Aditya Bhaskara,et al.  Detecting high log-densities: an O(n¼) approximation for densest k-subgraph , 2010, STOC '10.

[18]  Alkis Gotovos,et al.  Non-Monotone Adaptive Submodular Maximization , 2015, IJCAI.

[19]  Kaito Fujii,et al.  Beyond Adaptive Submodularity: Approximation Guarantees of Greedy Policy with Adaptive Submodularity Ratio , 2019, ICML.

[20]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[21]  David Cohn,et al.  Active Learning , 2010, Encyclopedia of Machine Learning.

[22]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Andreas Krause,et al.  Selecting Sequences of Items via Submodular Maximization , 2017, AAAI.

[24]  Morteza Zadimoghaddam,et al.  Submodular Maximization with Nearly Optimal Approximation, Adaptivity and Query Complexity , 2018, SODA.

[25]  Jure Leskovec,et al.  Inferring networks of diffusion and influence , 2010, KDD.

[26]  Huy L. Nguyen,et al.  Submodular Maximization with Nearly-optimal Approximation and Adaptivity in Nearly-linear Time , 2018, SODA.

[27]  Eric Balkanski,et al.  The adaptive complexity of maximizing a submodular function , 2018, STOC.

[28]  Alexandros G. Dimakis,et al.  Restricted Strong Convexity Implies Weak Submodularity , 2016, The Annals of Statistics.

[29]  Guy Kortsarz,et al.  On Choosing a Dense Subgraph (Extended Abstract) , 1993, FOCS 1993.

[30]  Olgica Milenkovic,et al.  Inhomogeneous Hypergraph Clustering with Applications , 2017, NIPS.

[31]  Andreas Krause,et al.  Streaming submodular maximization: massive data summarization on the fly , 2014, KDD.

[32]  Amit Chakrabarti,et al.  Submodular maximization meets streaming: matchings, matroids, and more , 2013, Math. Program..

[33]  Roy Schwartz,et al.  Online Submodular Maximization with Preemption , 2015, SODA.

[34]  M. Kendall.  A NEW MEASURE OF RANK CORRELATION , 1938 .

[35]  Stefanie Jegelka,et al.  Robust Budget Allocation Via Continuous Submodular Functions , 2017, Applied Mathematics & Optimization.

[36]  Abhimanyu Das,et al.  Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection , 2011, ICML.

[37]  Amin Karbasi,et al.  Do Less, Get More: Streaming Submodular Maximization with Subsampling , 2018, NeurIPS.

[38]  Huy L. Nguyen,et al.  The Power of Randomization: Distributed Submodular Maximization on Massive Datasets , 2015, ICML.

[39]  Morteza Zadimoghaddam,et al.  Randomized Composable Core-sets for Distributed Submodular Maximization , 2015, STOC.

[40]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[41]  Jeff A. Bilmes,et al.  Submodular Maximization via Gradient Ascent: The Case of Deep Submodular Functions , 2018, NeurIPS.

[42]  Francis Bach,et al.  Submodular functions: from discrete to continuous domains , 2015, Mathematical Programming.

[43]  Alexandros G. Dimakis,et al.  Streaming Weak Submodularity: Interpreting Neural Networks on the Fly , 2017, NIPS.

[44]  Pasin Manurangsi,et al.  Almost-polynomial ratio ETH-hardness of approximating densest k-subgraph , 2016, STOC.

[45]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[46]  Andreas Krause,et al.  Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization , 2010, J. Artif. Intell. Res..

[47]  Laurence A. Wolsey,et al.  An analysis of the greedy algorithm for the submodular set covering problem , 1982, Comb..

[48]  Andreas Krause,et al.  Budgeted Nonparametric Learning from Data Streams , 2010, ICML.

[49]  Ola Svensson,et al.  Beyond 1/2-Approximation for Submodular Maximization on Massive Data Streams , 2018, ICML.

[50]  Sergei Vassilvitskii,et al.  Fast greedy algorithms in mapreduce and streaming , 2013, SPAA.

[51]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[52]  Amin Karbasi,et al.  Gradient Methods for Submodular Maximization , 2017, NIPS.

[53]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[54]  Azarakhsh Malekian,et al.  Maximizing Sequence-Submodular Functions and its Application to Online Advertising , 2010, Manag. Sci..

[55]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[56]  Zheng Wen,et al.  Adaptive Submodular Maximization in Bandit Setting , 2013, NIPS.

[57]  George J. Pappas,et al.  Resilient monotone submodular function maximization , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[58]  Yisong Yue,et al.  Linear Submodular Bandits and their Application to Diversified Retrieval , 2011, NIPS.