Active learning of self-concordant like multi-index functions

We study the problem of actively learning a multi-index function of the form f(x) = g<sub>0</sub>(A<sub>0</sub>x) from its point evaluations, where A<sub>0</sub> ∈ ℝ<sup>k×d</sup> with k ≫ d. We build on the assumptions and techniques of an existing approach based on low-rank matrix recovery (Tyagi and Cevher, 2012). Specifically, by introducing an additional self- concordant like assumption on g0 and adapting the sampling scheme and its analysis accordingly, we provide a bound on the sampling complexity with a weaker dependence on d in the presence of additive Gaussian sampling noise. For example, under natural assumptions on certain other parameters, the dependence decreases from O(d<sup>3/2</sup>) to O(d<sup>¾</sup>).

[1]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[2]  Emmanuel J. Candès,et al.  Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements , 2011, IEEE Transactions on Information Theory.

[3]  Volkan Cevher,et al.  Active Learning of Multi-Index Function Models , 2012, NIPS.

[4]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[5]  Emmanuel J. Candès,et al.  Tight oracle bounds for low-rank matrix recovery from a minimal number of random measurements , 2010, ArXiv.

[6]  Y. Xia,et al.  A Multiple-Index Model and Dimension Reduction , 2008 .

[7]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[8]  I. Daubechies,et al.  Capturing Ridge Functions in High Dimensions from Point Queries , 2012 .

[9]  E. Candès.  Ridgelets: estimating with ridge functions , 2003 .

[10]  I. Johnstone,et al.  Projection-Based Approximation and a Duality with Kernel Methods , 1989 .

[11]  V. Cevher,et al.  Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods , 2013, 1310.1826.

[12]  Jan Vybíral,et al.  Learning Functions of Few Arbitrary Linear Parameters in High Dimensions , 2010, Found. Comput. Math..

[13]  Allan Pinkus,et al.  Approximation theory of the MLP model in neural networks , 1999, Acta Numerica.

[14]  Andreas Krause,et al.  High-Dimensional Gaussian Process Bandits , 2013, NIPS.

[15]  Francis R. Bach,et al.  Self-concordant analysis for logistic regression , 2009, ArXiv.