Heuristics for semi-external depth first search on directed graphs

Computing the strong components of a directed graph is an essential operation for a basic structural analysis of it. This problem can be solved by twice running a depth-first search (DFS). In an external setting, in which all data can no longer be stored in the main memory, the DFS problem is unsolved so far. Assuming that node-related data can be stored internally, semi-external computing does not make the problem substantially easier. Considering the definite need to analyze very large graphs, we have developed a set of heuristics which together allow the performance of semi-external DFS for directed graphs in practice. The heuristics have been applied to graphs with very different graph properties, including "web graphs" as described in the most recent literature and some large call graphs from ATT. Depending on the graph structure, the program is between 10 and 200 times faster than the best alternative, a factor that will further increase with future technological developments.

[1]  Vijay Kumar,et al.  Improved algorithms and data structures for solving graph problems in external memory , 1996, Proceedings of SPDP '96: 8th IEEE Symposium on Parallel and Distributed Processing.

[2]  Alok Aggarwal,et al.  The input/output complexity of sorting and related problems , 1988, CACM.

[3]  Siavash Vahdati Daneshmand,et al.  Improved algorithms for the Steiner problem in networks , 2001, Discret. Appl. Math..

[4]  Ulrich Meyer,et al.  On External-Memory Planar Depth First Search , 2001, J. Graph Algorithms Appl..

[5]  Suresh Venkatasubramanian,et al.  On external memory graph traversal , 2000, SODA '00.

[6]  Robert E. Tarjan,et al.  A data structure for dynamic trees , 1981, STOC '81.

[7]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[8]  Norbert Zeh,et al.  External Memory Algorithms for Outerplanar Graphs , 1999, ISAAC.

[9]  Ulrich Meyer.  External memory BFS on undirected graphs with bounded degree , 2001, SODA '01.

[10]  Thorsten Koch,et al.  Solving Steiner tree problems in graphs to optimality , 1998, Networks.

[11]  Norbert Zeh,et al.  I/O-Efficient Algorithms for Graphs of Bounded Treewidth , 2001, SODA '01.

[12]  Josep Díaz Cort,et al.  Random geometric problems on $[0,1]^2$ , 1998 .

[13]  Norbert Zeh,et al.  I/O-optimal algorithms for planar graphs using separators , 2002, SODA '02.

[14]  Andrei Z. Broder,et al.  Graph structure in the Web , 2000, Comput. Networks.

[15]  Robert E. Tarjan,et al.  Self-adjusting binary search trees , 1985, JACM.

[16]  Alok Aggarwal,et al.  A random NC algorithm for depth first search , 1987, Comb..

[17]  Hans Jürgen Prömel,et al.  The Steiner Tree Problem , 2002 .

[18]  Maria J. Serna,et al.  Random Geometric Problems on [0, 1]² , 1998, RANDOM.

[19]  Robert Sedgewick,et al.  Average case analysis of graph-searching algorithms , 1990 .

[20]  Alan M. Frieze,et al.  A General Model of Undirected Web Graphs , 2001, ESA.

[21]  Edward F. Grove,et al.  External-memory graph algorithms , 1995, SODA '95.