Approximate Supermodularity Bounds for Experimental Design

This work provides performance guarantees for the greedy solution of experimental design problems. In particular, it focuses on A- and E-optimal designs, for which typical guarantees do not apply since the mean-square error and the maximum eigenvalue of the estimation error covariance matrix are not supermodular. To do so, it leverages the concept of approximate supermodularity to derive non-asymptotic worst-case suboptimality bounds for these greedy solutions. These bounds reveal that as the SNR of the experiments decreases, these cost functions behave increasingly as supermodular functions. As such, greedy A- and E-optimal designs approach (1-1/e)-optimality. These results reconcile the empirical success of greedy experimental design with the non-supermodularity of the A- and E-optimality criteria.

[1]  Andreas Krause,et al.  Guarantees for Greedy Maximization of Non-submodular Functions with Applications , 2017, ICML.

[2]  Michael I. Jordan,et al.  Robust design of biological experiments , 2005, NIPS.

[3]  Alexandros G. Dimakis,et al.  Restricted Strong Convexity Implies Weak Submodularity , 2016, The Annals of Statistics.

[4]  Allan Borodin,et al.  Weakly Submodular Functions , 2014, ArXiv.

[5]  Stratis Ioannidis,et al.  Budget Feasible Mechanisms for Experimental Design , 2013, LATIN.

[6]  Guillaume Sagnol,et al.  Approximation of a maximum-submodular-coverage problem involving spectral functions, with application to experimental designs , 2010, Discret. Appl. Math..

[7]  Jinbo Bi,et al.  Active learning via transductive experimental design , 2006, ICML.

[8]  Andreas Krause,et al.  Submodular Function Maximization , 2014, Tractability.

[9]  Stephen P. Boyd,et al.  Sensor Selection via Convex Optimization , 2009, IEEE Transactions on Signal Processing.

[10]  Andreas Krause,et al.  Submodular Dictionary Selection for Sparse Representation , 2010, ICML.

[11]  Yaron Singer,et al.  Maximization of Approximately Submodular Functions , 2016, NIPS.

[12]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[13]  Yoshikazu Washizawa,et al.  Subset kernel principal component analysis , 2009, 2009 IEEE International Workshop on Machine Learning for Signal Processing.

[14]  Abhimanyu Das,et al.  Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection , 2011, ICML.

[15]  Aarti Singh,et al.  On Computationally Tractable Selection of Experiments in Measurement-Constrained Regression Models , 2016, J. Mach. Learn. Res..

[16]  Sundeep Prabhakar Chepuri,et al.  Sensor Selection for Estimation with Correlated Measurement Noise , 2015, IEEE Transactions on Signal Processing.

[17]  Alejandro Ribeiro,et al.  Near-optimality of greedy set selection in the sampling of graph signals , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[18]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[19]  Maxim Sviridenko,et al.  Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee , 2004, J. Comb. Optim..

[20]  John Lygeros,et al.  On Submodularity and Controllability in Complex Dynamical Networks , 2014, IEEE Transactions on Control of Network Systems.

[21]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[22]  Adams Wei Yu,et al.  On Computationally Tractable Selection of Experiments in Regression Models , 2016 .

[23]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[24]  Francis R. Bach,et al.  Learning with Submodular Functions: A Convex Optimization Perspective , 2011, Found. Trends Mach. Learn..

[25]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[26]  Abhimanyu Das,et al.  Algorithms for subset selection in linear regression , 2008, STOC.

[27]  S. R. Searle,et al.  On Deriving the Inverse of a Sum of Matrices , 1981 .

[28]  F. Pukelsheim.  Optimal Design of Experiments , 1993 .

[29]  R. Bhatia.  Matrix Analysis , 1996 .

[30]  Guillaume Sagnol,et al.  Computing Optimal Designs of multiresponse Experiments reduces to Second-Order Cone Programming , 2009, 0912.5467.

[31]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .