Geometric Tools for Algorithms.

Abstract : Our thesis is that a geometric perspective yields insights into the structure of fundamental problems, and thereby suggests efficient algorithms for them. As evidence we develop new geometric models and general-purpose tools for removing outliers from numeric data, reducing dimensionality, and counting combinatorial sets. Then we apply these techniques to a set of old problems to obtain polynomial-time algorithms. These include: (1) learning noisy linear-threshold functions (half-spaces), (2) learning the intersection of halfspaces, (3) clustering text corpora, and (4) counting lattice points in a convex body. We supplement some of our theorems with experimental studies.

[1]  I. J. Schoenberg,et al.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[2]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[3]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[4]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[5]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[6]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[7]  Leslie G. Valiant,et al.  The Complexity of Computing the Permanent , 1979, Theor. Comput. Sci..

[8]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[9]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[10]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[11]  P. Diaconis,et al.  Testing for independence in a two-way table , 1985 .

[12]  Leslie G. Valiant,et al.  Random Generation of Combinatorial Structures from a Uniform Distribution , 1986, Theor. Comput. Sci..

[13]  Prabhakar Raghavan,et al.  Randomized rounding: A technique for provably good algorithms and algorithmic proofs , 1985, Comb..

[14]  James A. Anderson,et al.  Neurocomputing: Foundations of Research , 1988 .

[15]  Susan T. Dumais,et al.  Using latent semantic analysis to improve information retrieval , 1988, CHI 1988.

[16]  Nimrod Megiddo,et al.  On the complexity of polyhedral separability , 1988, Discret. Comput. Geom..

[17]  Peter Frankl,et al.  The Johnson-Lindenstrauss lemma and the sphericity of some graphs , 1987, J. Comb. Theory B.

[18]  Mark Jerrum,et al.  Approximating the Permanent , 1989, SIAM J. Comput..

[19]  Wolfgang Maass,et al.  On the complexity of learning from counterexamples , 1989, 30th Annual Symposium on Foundations of Computer Science.

[20]  Miklós Simonovits,et al.  The mixing rate of Markov chains, an isoperimetric inequality, and computing the volume , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[21]  Philip M. Long,et al.  Composite geometric concepts and polynomial predictability , 1990, COLT '90.

[22]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[23]  ERIC B. BAUM,et al.  On learning a union of half spaces , 1990, J. Complex..

[24]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[25]  Eric B. Baum,et al.  Polynomial time algorithms for learning neural nets , 1990, Annual Conference Computational Learning Theory.

[26]  Susan T. Dumais,et al.  Improving the retrieval of information from external sources , 1991 .

[27]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[28]  Yoav Freund,et al.  An improved boosting algorithm and its implications on learning complexity , 1992, COLT '92.

[29]  Tom Bylander.  Polynomial learnability of linear threshold approximations , 1993, COLT '93.

[30]  Michael W. Berry,et al.  SVDPACKC (Version 1.0) User''s Guide , 1993 .

[31]  Avrim Blum,et al.  Learning an intersection of k halfspaces over a uniform distribution , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[32]  Javed A. Aslam,et al.  General bounds on statistical query learning and PAC learning with noise via hypothesis boosting , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[33]  Martin E. Dyer,et al.  A Mildly Exponential Time Algorithm for Approximating the Number of Solutions to a Multidimensional Knapsack Problem , 1993, Combinatorics, Probability and Computing.

[34]  Edoardo Amaldi,et al.  From finding maximum feasible subsystems of linear systems to feedforward neural network design , 1994 .

[35]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[36]  Javed A. Aslam,et al.  Improved Noise-Tolerant Learning and Generalized Statistical Queries , 1994 .

[37]  Tom Bylander,et al.  Learning linear threshold functions in the presence of classification noise , 1994, COLT '94.

[38]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[39]  Santosh S. Vempala,et al.  Sampling lattice points , 1997, STOC '97.

[40]  Santosh S. Vempala,et al.  A random sampling based algorithm for learning the intersection of half-spaces , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[41]  Edith Cohen,et al.  Learning noisy perceptrons by a perceptron in polynomial time , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[42]  Jon M. Kleinberg,et al.  Two algorithms for nearest-neighbor search in high dimensions , 1997, STOC '97.

[43]  M. Kearns.  Efficient noise-tolerant learning from statistical queries , 1998, JACM.

[44]  Santosh S. Vempala,et al.  Latent semantic indexing: a probabilistic analysis , 1998, PODS '98.

[45]  Alan M. Frieze,et al.  A Polynomial-Time Algorithm for Learning Noisy Linear Threshold Functions , 1996, Algorithmica.