The MapReduce-based approach to improve the shortest path computation in large-scale road networks: the case of A* algorithm

This paper deals with an efficient parallel and distributed framework for intensive computation with A* algorithm based on MapReduce concept. The A* algorithm is one of the most popular graph traversal algorithm used in route guidance. It requires exponential time computation and very costly hardware to compute the shortest path on large-scale networks. Thus, it is necessary to reduce the time complexity while exploiting a low cost commodity hardwares. To cope with this situation, we propose a novel approach that reduces the A* algorithm into a set of Map and Reduce tasks for running the path computation on Hadoop MapReduce framework. An application on real road networks illustrates the feasibility and reliability of the proposed framework. The experiments performed on a 6-node Hadoop cluster proves that the proposed approach outperforms A* algorithm and achieves significant gain in terms of computation time.

[1]  K. Subramani,et al.  A faster algorithm for the single source shortest path problem with few distinct positive lengths , 2010, J. Discrete Algorithms.

[2]  Stuart J. Russell.  Efficient Memory-Bounded Search Methods , 1992, ECAI.

[3]  Sabeur Aridhi,et al.  SHORTEST PATH RESOLUTION USING HADOOP , 2014 .

[4]  Eric A. Hansen,et al.  Breadth-first heuristic search , 2004, Artif. Intell..

[5]  Sabeur Aridhi,et al.  A MapReduce-based approach for shortest path problem in large-scale networks , 2015, Eng. Appl. Artif. Intell..

[6]  Jae-Gil Lee,et al.  Parallel community detection on large graphs with MapReduce and GraphChi , 2016, Data Knowl. Eng..

[7]  Andrew V. Goldberg,et al.  Shortest paths algorithms: Theory and experimental evaluation , 1994, SODA '94.

[8]  Carlo Curino,et al.  Apache Hadoop YARN: yet another resource negotiator , 2013, SoCC.

[9]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[10]  Dominique Lavenier,et al.  All-Pairs Shortest Path algorithms for planar graph for GPU-accelerated clusters , 2015, J. Parallel Distributed Comput..

[11]  Sabeur Aridhi,et al.  Density-based data partitioning strategy to approximate large-scale subgraph mining , 2012, Inf. Syst..

[12]  Takashi Washio,et al.  An Apriori-Based Algorithm for Mining Frequent Substructures from Graph Data , 2000, PKDD.

[13]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[14]  Jonathan Cohen,et al.  Graph Twiddling in a MapReduce World , 2009, Computing in Science & Engineering.

[15]  Steven Skiena,et al.  The Algorithm Design Manual , 2020, Texts in Computer Science.

[16]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[17]  Richard Bellman,et al.  ON A ROUTING PROBLEM , 1958 .

[18]  Adi Botea,et al.  Near Optimal Hierarchical Path-Finding , 2004, J. Game Dev..

[19]  Richard L. Church,et al.  Finding shortest paths on real road networks: the case for A* , 2009, Int. J. Geogr. Inf. Sci..

[20]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[21]  Lu Xiong,et al.  The Research of Dynamic Shortest Path Based on Cloud Computing , 2016, 2016 12th International Conference on Computational Intelligence and Security (CIS).

[22]  Laurence R. Rilett,et al.  Heuristic shortest path algorithms for transportation applications: State of the art , 2006, Comput. Oper. Res..

[23]  A. Schrijver.  On the History of the Shortest Path Problem , 2012 .

[24]  GhemawatSanjay,et al.  The Google file system , 2003 .

[25]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[26]  Shimon Even,et al.  Graph Algorithms: Contents , 2011 .

[27]  Shimon Even,et al.  Graph Algorithms , 1979 .

[28]  Y. Q. Liu,et al.  An improved Dijkstra's shortest path algorithm for sparse network , 2007, Appl. Math. Comput..

[29]  Brahim Aghezzaf,et al.  The MapReduce-based approach to improve vehicle controls on big traffic events , 2017, 2017 International Colloquium on Logistics and Supply Chain Management (LOGISTIQUA).

[30]  Andrew V. Goldberg,et al.  Computing the shortest path: A search meets graph theory , 2005, SODA '05.

[31]  Steven J. Plimpton,et al.  MapReduce in MPI for Large-scale graph algorithms , 2011, Parallel Comput..

[32]  Guy Even,et al.  Graph Algorithms: Preface to the Second Edition , 2011 .

[33]  Robert E. Tarjan,et al.  Fibonacci heaps and their uses in improved network optimization algorithms , 1984, JACM.

[34]  Thomas H. Cormen,et al.  Introduction to algorithms [2nd ed.] , 2001 .