Utilizing Geometric Anomalies of High Dimension: When Complexity Makes Computation Easier

Just as a busy kitchen can be more efficient than an idle one, Kleinrock showed 35 years ago that heavily used networks admit simple heuristic approximations with excellent quantitative accuracy. We describe a number of different examples in which having many parameters actually facilitates computation and we suggest connections with geometric phenomena in high-dimensional spaces. It seems that in several interesting and quite general situations, dimensionality may be a blessing in disguise provided that some suitable form of computing is used which can deal with it.

[1]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[2]  Jeffrey D. Ullman,et al.  Worst-Case Performance Bounds for Simple One-Dimensional Packing Algorithms , 1974, SIAM J. Comput..

[3]  Leonard Kleinrock,et al.  Communication Nets: Stochastic Message Flow and Delay , 1964 .

[4]  Paul C. Kainen,et al.  Quasiorthogonal dimension of euclidean spaces , 1993 .

[5]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[6]  N. J. A. Sloane,et al.  Sphere Packings, Lattices and Groups , 1987, Grundlehren der mathematischen Wissenschaften.

[7]  R Courant,et al.  Differential And Integral Calculus Vol-ii , 1936 .

[8]  J. Spencer.  Ten lectures on the probabilistic method , 1987 .

[9]  Frank Thomson Leighton,et al.  Some unexpected expected behavior results for bin packing , 1984, STOC '84.

[10]  Walter Böhm,et al.  Expected Wire Length Between Two Randomly Chosen Terminals (David M. Lazoff) , 1996, SIAM Rev..

[11]  Leonid Levant,et al.  Induction in geometry , 1963 .

[12]  M Desmurget,et al.  Postural and synergic control for three-dimensional movements of reaching and grasping. , 1995, Journal of neurophysiology.

[13]  Vladik Kreinovich,et al.  Estimates of the Number of Hidden Units and Variation with Respect to Half-Spaces , 1997, Neural Networks.

[14]  Alan T. Sherman,et al.  Expected Wire Length between Two Randomly Chosen Terminals , 1995, SIAM Rev..

[15]  Richard W. Hamming,et al.  Coding and Information Theory , 2018, Feynman Lectures on Computation.

[16]  On the immersion of digraphs in cubes , 1974 .

[17]  David S. Johnson,et al.  Computers and In stractability: A Guide to the Theory of NP-Completeness. W. H Freeman, San Fran , 1979 .

[18]  Paul C. Kainen,et al.  A geometric method to obtain error-correcting classification by neural networks with fewer hidden units , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).