论文信息 - Sampling algorithms and coresets for ℓp regression

Sampling algorithms and coresets for ℓp regression

The ℓp regression problem takes as input a matrix A ∈ ℝn, a vector b ∈ ℝn, and a number p ∈ [1, ∞), and it returns as output a number Z and a vector xOPT ∈ ℝd such that Z = minx∈ℝd ||Ax - b||p = ||AxOPT - b||p. In this paper, we construct coresets and obtain an efficient two-stage sampling-based approximation algorithm for the very overconstrained (n ≫ d) version of this classical problem, for all p ∈ [1, ∞). The first stage of our algorithm non-uniformly samples &rcirc;1 = O(36pdmax{p/2+1, p}+1) rows of A and the corresponding elements of b, and then it solves the lp regression problem on the sample; we prove this is an 8-approximation. The second stage of our algorithm uses the output of the first stage to resample &rcirc;1/ε2 constraints, and then it solves the lp regression problem on the new sample; we prove this is a (1 + ε)-approximation. Our algorithm unifies, improves upon, and extends the existing algorithms for special cases of ℓp regression, namely p = 1,2 [10, 13]. In course of proving our result, we develop two concepts--well-conditioned bases and subspace-preserving sampling--that are of independent interest.

[1] L. M. M.-T.. Theory of Probability , 1929, Nature.

[2] H. Jeffreys,et al. The Theory of Probability , 1896 .

[3] Angus E. Taylor. A geometric theorem and its application to biorthogonal systems , 1947 .

[4] László Lovász,et al. Algorithmic theory of numbers, graphs and convexity , 1986, CBMS-NSF regional conference series in applied mathematics.

[5] G. Schechtman. More on embedding subspaces of $L_p$ in $l^n_r$ , 1987 .

[6] J. Lindenstrauss,et al. Approximation of zonoids by zonotopes , 1989 .

[7] M. Talagrand. Embedding subspaces of ₁ into ^{}₁ , 1990 .

[8] M. Talagrand. Embedding Subspaces of L 1 into l N 1 , 1990 .

[9] M. Talagrand. Embedding subspaces of L1 into l1N , 1990 .

[10] P. Wojtaszczyk. Banach Spaces For Analysts: Preface , 1991 .

[11] Gideon Schechtman Of Rehovot,et al. Embedding Subspaces of L P Intò N P , 0 < P < 1 , 1999 .

[12] Trevor Hastie,et al. The Elements of Statistical Learning , 2001 .

[13] Terri L. Moore,et al. Regression Analysis by Example , 2001, Technometrics.

[14] Jiri Matousek,et al. Lectures on discrete geometry , 2002, Graduate texts in mathematics.

[15] Kenneth L. Clarkson,et al. Smaller core-sets for balls , 2003, SODA '03.

[16] Andreas Maurer. A bound on the deviation probability for sums of non-negative random variables. , 2003 .

[17] Sariel Har-Peled,et al. On coresets for k-means and k-median clustering , 2004, STOC '04.

[18] Baruch Awerbuch,et al. Adaptive routing with end-to-end feedback: distributed learning and geometric approaches , 2004, STOC '04.

[19] Pankaj K. Agarwal,et al. Approximating extent measures of points , 2004, JACM.

[20] Jon M. Kleinberg,et al. Using mixture models for collaborative filtering , 2004, STOC '04.

[21] K. Clarkson. Subgradient and sampling algorithms for l1 regression , 2005, SODA '05.

[22] Tamás Sarlós,et al. Improved Approximation Algorithms for Large Matrices via Random Projections , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[23] Amos Fiat,et al. Coresets forWeighted Facilities and Their Applications , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[24] Dan Feldman. Coresets for Weighted Facilities and Their Applications , 2006 .

[25] Stephen P. Boyd,et al. Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[26] Bernard Chazelle,et al. Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform , 2006, STOC '06.

[27] S. Muthukrishnan,et al. Sampling algorithms for l2 regression and applications , 2006, SODA '06.

[28] Kasturi R. Varadarajan,et al. Geometric Approximation via Coresets , 2007 .

[29] M. Day. POLYGONS CIRCUMSCRIBED ABOUT CLOSED CONVEX CURVES , 2010 .