Parallelism in Randomized Incremental Algorithms

In this paper we show that most sequential randomized incremental algorithms are in fact parallel. We consider several random incremental algorithms including algorithms for comparison sorting and Delaunay triangulation; linear programming, closest pair, and smallest enclosing disk in constant dimensions; as well as least-element lists and strongly connected components on graphs. We analyze the dependence between iterations in an algorithm, and show that the dependence structure is shallow for all of the algorithms, implying high parallelism. We identify three types of dependences found in the algorithms studied and present a framework for analyzing each type of algorithm. Using the framework gives work-efficient polylogarithmic-depth parallel algorithms for most of the problems that we study. Some of these algorithms are straightforward (e.g., sorting and linear programming), while others are more novel and require more effort to obtain the desired bounds (e.g., Delaunay triangulation and strongly connected components). The most surprising of these results is for planar Delaunay triangulation for which the incremental approach is by far the most commonly used in practice, but for which it was not previously known whether it is theoretically efficient in parallel.

[1]  Nancy M. Amato,et al.  Parallel algorithms for higher-dimensional convex hulls , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[2]  Diego R. Llanos,et al.  Parallelizing 2 D-Convex Hulls on clusters : Sorting matters , .

[3]  Raimund Seidel,et al.  Small-dimensional linear programming and convex hulls made easy , 1991, Discret. Comput. Geom..

[4]  Mihalis Yannakakis,et al.  High-probability parallel transitive closure algorithms , 1990, SPAA '90.

[5]  Sivasankaran Rajamanickam,et al.  BFS and Coloring-Based Parallel Algorithms for Strongly Connected Components and Related Problems , 2014, 2014 IEEE 28th International Parallel and Distributed Processing Symposium.

[6]  Pat Morin.  Optimal Randomized Parallel Algorithms for Computational Geometry , 2007 .

[7]  References , 1971 .

[8]  Dahlia Malkhi,et al.  Efficient distributed approximation algorithms via probabilistic tree embeddings , 2008, PODC '08.

[9]  Nimrod Megiddo,et al.  Linear-time algorithms for linear programming in R3 and related problems , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[10]  John H. Reif,et al.  Erratum: Optimal Parallel Randomized Algorithms for Three-Dimensional Convex Hulls and Related Problems , 1994, SIAM J. Comput..

[11]  Guy E. Blelloch,et al.  Design and Implementation of a Practical Parallel Delaunay Algorithm , 1999, Algorithmica.

[12]  Nimrod Megiddo,et al.  Linear-Time Algorithms for Linear Programming in R^3 and Related Problems , 1982, FOCS.

[13]  Guy E. Blelloch,et al.  Internally deterministic parallel algorithms can be fast , 2012, PPoPP '12.

[14]  Robin Sibson,et al.  Computing Dirichlet Tessellations in the Plane , 1978, Comput. J..

[15]  Guy E. Blelloch,et al.  Efficient Construction of Probabilistic Tree Embeddings , 2016, ICALP.

[16]  Charles E. Leiserson,et al.  Ordering heuristics for parallel graph coloring , 2014, SPAA.

[17]  Keshav Pingali,et al.  The tao of parallelism in algorithms , 2011, PLDI '11.

[18]  Herbert Edelsbrunner,et al.  Geometry and Topology for Mesh Generation , 2001, Cambridge monographs on applied and computational mathematics.

[19]  Xiaotie Deng.  An Optimal Parallel Algorithm for Linear Programming in the Plane , 1990, Inf. Process. Lett..

[20]  Guy E. Blelloch,et al.  Parallel Write-Efficient Algorithms and Data Structures for Computational Geometry , 2018, SPAA.

[21]  Guy E. Blelloch,et al.  Sequential Random Permutation, List Contraction and Tree Contraction are Highly Parallel , 2015, SODA.

[22]  Guy E. Blelloch,et al.  Parallel Shortest Paths Using Radius Stepping , 2016, SPAA.

[23]  Richard Cole,et al.  Parallel merge sort , 1988, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[24]  Dahlia Malkhi,et al.  Efficient distributed approximation algorithms via probabilistic tree embeddings , 2008, PODC '08.

[25]  Nancy M. Amato,et al.  Efficient , Reachability-based , Parallel Algorithms for Finding Strongly Connected Components ∗ , 2015 .

[26]  Emo Welzl,et al.  Smallest enclosing disks (balls and ellipsoids) , 1991, New Results and New Trends in Computer Science.

[27]  Klaus Sutner.  Probabilistic Algorithms , 2017 .

[28]  Guy E. Blelloch,et al.  Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable , 2018, SPAA.

[29]  R. Seidel.  Backwards Analysis of Randomized Geometric Algorithms , 1993 .

[30]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[31]  Jean-Daniel Boissonnat,et al.  On the Randomized Construction of the Delaunay Tree , 1993, Theor. Comput. Sci..

[32]  S. Janson.  Tail bounds for sums of geometric and exponential variables , 2017, 1709.08157.

[33]  Guy E. Blelloch,et al.  Engineering a compact parallel delaunay algorithm in 3D , 2006, SCG '06.

[34]  Herbert Edelsbrunner,et al.  Incremental topological flipping works for regular triangulations , 1992, SCG '92.

[35]  Jinhui Xu,et al.  Two-variable linear programming in parallel , 1998, Comput. Geom..

[36]  Arturo Gonzalez-Escribano,et al.  Parallelization alternatives and their performance for the convex hull problem , 2006 .

[37]  Richard P. Brent,et al.  The Parallel Evaluation of General Arithmetic Expressions , 1974, JACM.

[38]  Michael T. Goodrich,et al.  Bounded-Independence Derandomization of Geometric Partitioning with Applications to Parallel Fixed-Dimensional Linear Programming , 1997, Discret. Comput. Geom..

[39]  Guy E. Blelloch,et al.  Parallel probabilistic tree embeddings, k-median, and buy-at-bulk network design , 2012, SPAA '12.

[40]  Sariel Har-Peled.  Geometric Approximation Algorithms , 2011 .

[41]  Uzi Vishkin,et al.  Thinking in Parallel: Some Basic Data-Parallel Algorithms and Techniques , 2008 .

[42]  Jeremy T. Fineman.  Nearly work-efficient parallel algorithm for digraph reachability , 2017, STOC.

[43]  Sanguthevar Rajasekaran,et al.  Optimal and Sublogarithmic Time Randomized Parallel Sorting Algorithms , 1989, SIAM J. Comput..

[44]  Martin E. Dyer,et al.  A parallel algorithm for linear programming in fixed dimension , 1995, SCG '95.

[45]  Dimitris S. Papailiopoulos,et al.  Parallel Correlation Clustering on Big Graphs , 2015, NIPS.

[46]  Thomas H. Spencer,et al.  Time-work tradeoffs for parallel algorithms , 1997, JACM.

[47]  Le Song,et al.  Scalable Influence Estimation in Continuous-Time Diffusion Networks , 2013, NIPS.

[48]  Bruce Hendrickson,et al.  A Divide-and-conquer Algorithm for Identifying Strongly Connected Components ? , 2003 .

[49]  Uzi Vishkin,et al.  Towards a theory of nearly constant time parallel algorithms , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[50]  Samir Khuller,et al.  A Simple Randomized Sieve Algorithm for the Closest-Pair Problem , 1995, Inf. Comput..

[51]  Sarita V. Adve,et al.  Parallel programming must be deterministic by default , 2009 .

[52]  Kunle Olukotun,et al.  STAMP: Stanford Transactional Applications for Multi-Processing , 2008, 2008 IEEE International Symposium on Workload Characterization.

[53]  Lubos Brim,et al.  Computing Strongly Connected Components in Parallel on CUDA , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[54]  S. Sitharama Iyengar,et al.  Introduction to parallel algorithms , 1998, Wiley series on parallel and distributed computing.

[55]  Michael T. Goodrich,et al.  Fixed-dimensional parallel linear programming via relative ε-approximations , 1996, SODA '96.

[56]  Noga Alon,et al.  Parallel linear programming in fixed dimension almost surely in constant time , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[57]  Kunle Olukotun,et al.  On fast parallel detection of strongly connected components (SCC) in small-world graphs , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[58]  Danny Z. Chen,et al.  Deterministic Parallel Computational Geometry , 2000, Handbook of Computational Geometry.

[59]  Edith Cohen,et al.  Efficient estimation algorithms for neighborhood variance and other moments , 2004, SODA '04.

[60]  Dan Alistarh,et al.  Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms , 2018, PODC.

[61]  Guy E. Blelloch,et al.  Parallel Algorithms for Asymmetric Read-Write Costs , 2016, SPAA.

[62]  Edith Cohen,et al.  Size-Estimation Framework with Applications to Transitive Closure and Reachability , 1997, J. Comput. Syst. Sci..

[63]  Ketan Mulmuley,et al.  Computational geometry : an introduction through randomized algorithms , 1993 .

[64]  Diego R. Llanos Ferraris,et al.  Meseta: a new scheduling strategy for speculative parallelization of randomized incremental algorithms , 2005, 2005 International Conference on Parallel Processing Workshops (ICPPW'05).

[65]  Guy E. Blelloch,et al.  Efficient Algorithms with Asymmetric Read and Write Costs , 2015, ESA.

[66]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[67]  Philip N. Klein,et al.  A Randomized Parallel Algorithm for Single-Source Shortest Paths , 1997, J. Algorithms.

[68]  Guy E. Blelloch,et al.  Brief announcement: the problem based benchmark suite , 2012, SPAA '12.

[69]  Satish Rao,et al.  A tight bound on approximating arbitrary metrics by tree metrics , 2003, STOC '03.

[70]  Diego R. Llanos Ferraris,et al.  Speculative Parallelization of a Randomized Incremental Convex Hull Algorithm , 2004, ICCSA.

[71]  Nimrod Megiddo,et al.  A deterministic poly(log log N)-time N-processor algorithm for linear programming in fixed dimension , 1992, STOC '92.

[72]  John H. Reif,et al.  Optimal randomized parallel algorithms for computational geometry , 2005, Algorithmica.

[73]  A. Vacavant,et al.  Reconstructions of Noisy Digital Contours with Maximal Primitives Based on Multi-Scale/Irregular Geometric Representation and Generalized Linear Programming , 2017 .

[74]  Guy E. Blelloch,et al.  Greedy sequential maximal independent set and matching are parallel on average , 2012, SPAA '12.

[75]  Kenneth L. Clarkson,et al.  Applications of random sampling in computational geometry, II , 1988, SCG '88.

[76]  Leonidas J. Guibas,et al.  Randomized incremental construction of Delaunay and Voronoi diagrams , 1990, Algorithmica.

[77]  Warren Schudy,et al.  Finding strongly connected components in parallel using O(log2n) reachability queries , 2008, SPAA '08.

[78]  Michiel H. M. Smid,et al.  Simple Randomized Algorithms for Closest Pair Problems , 1995, Nord. J. Comput..

[79]  Torben Hagerup.  Fast Parallel Generation of Random Permutations , 1991, ICALP.

[80]  John H. Reif,et al.  Depth-First Search is Inherently Sequential , 1985, Inf. Process. Lett..

[81]  Richard Cole,et al.  A nearly optimal deterministic parallel Voronoi diagram algorithm , 1996, Algorithmica.

[82]  Guy E. Blelloch,et al.  A Top-Down Parallel Semisort , 2015, SPAA.

[83]  R. Scopigno,et al.  Parallel 3D Delaunay Triangulation , 1993, Comput. Graph. Forum.

[84]  Guy E. Blelloch,et al.  Sorting with Asymmetric Read and Write Costs , 2015, SPAA.