Performance characterization of reinforcement learning-enabled evolutionary algorithms for integrated school bus routing and scheduling problem

Abstract Bi-objective school bus scheduling optimization problem that is a subset of vehicle fleet scheduling problem is focused in this paper. In the literature, school bus routing and scheduling problem is proven to be an NP-Hard problem. The processed data supplied by our framework is utilized to search a near-optimum schedule with the aid of reinforcement learning by evolutionary algorithms. They are named as reinforcement learning-enabled genetic algorithm (RL-enabled GA), reinforcement learning-enabled particle swarm optimization algorithm (RL-enabled PSO), and reinforcement learning-enabled ant colony optimization algorithm (RL-enabled ACO). In this paper, the performance characterization of reinforcement learning-enabled evolutionary algorithms for integrated school bus routing and scheduling problem is investigated. The efficiency of the conventional algorithms is improved, and the near-optimal schedule is achieved significantly in a shorter duration with the active guidance of the reinforcement learning algorithm. We attempt to carry out extensive performance evaluation and conducted experiments on a geospatial dataset comprising road networks, trip trajectories of buses, and the address of students. The conventional and reinforcement learning integrated algorithms are improving the travel time of buses and the students. More than 50% saving by the conventional and the reinforcement learning-enabled ant colony optimization algorithm compared to the constructive heuristic algorithm is achieved from 92nd and 54th iterations, respectively. Similarly, the saving by the conventional and the reinforcement learning-enabled genetic algorithm is 41.34% at 500th iterations and more than 50% improvement from 281st iterations, respectively. Lastly, more than 10% saving by the conventional and the reinforcement learning-enabled particle swarm algorithm is achieved from 432nd and 28th iterations, respectively.

[1]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[2]  Alireza Goli,et al.  Coordination policy for production and delivery scheduling in the closed loop supply chain , 2018, Prod. Eng..

[3]  Juan José Salazar González,et al.  Solving school bus routing using the multiple vehicle traveling purchaser problem: A branch-and-cut approach , 2012, Comput. Oper. Res..

[4]  Alireza Goli,et al.  An integrated disaster relief model based on covering tour using hybrid Benders decomposition and variable neighborhood search: Application in the Iranian context , 2019, Comput. Ind. Eng..

[5]  Paul H. Calamai,et al.  A multi-objective optimization approach to urban school bus routing: Formulation and solution method , 1995 .

[6]  Yongling Zheng,et al.  On the convergence analysis and parameter selection in particle swarm optimization , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[7]  Arun Kumar Sangaiah,et al.  Robust optimization and mixed-integer linear programming model for LNG supply chain planning problem , 2020, Soft Comput..

[8]  Fuzhong Nian,et al.  An Adaptive Particle Swarm Optimization Algorithm Based on Directed Weighted Complex Network , 2014 .

[9]  A. E. Eiben,et al.  A Generic Approach to Parameter Control , 2012, EvoApplications.

[10]  Yoshikazu Fukuyama,et al.  Practical distribution state estimation using hybrid particle swarm optimization , 2001, 2001 IEEE Power Engineering Society Winter Meeting. Conference Proceedings (Cat. No.01CH37194).

[11]  Frits C. R. Spieksma,et al.  A metaheuristic for the school bus routing problem with bus stop selection , 2013, Eur. J. Oper. Res..

[12]  Ali Mostafaeipour,et al.  Prediction of air travel demand using a hybrid artificial neural network (ANN) with Bat and Firefly algorithms: a case study , 2018, The Journal of Supercomputing.

[13]  Thomas Stützle,et al.  A Comparison Between ACO Algorithms for the Set Covering Problem , 2004, ANTS Workshop.

[14]  Jairo R. Montoya-Torres,et al.  Solving of school bus routing problem by ant colony optimization , 2012 .

[15]  Monique Snoeck,et al.  Classification With Ant Colony Optimization , 2007, IEEE Transactions on Evolutionary Computation.

[16]  Gerhard-Wilhelm Weber,et al.  Multi-objective Aggregate Production Planning Model Considering Overtime and Outsourcing Options Under Fuzzy Seasonal Demand , 2019, Lecture Notes in Mechanical Engineering.

[17]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[18]  Joaquín A. Pacheco,et al.  Bi-Objective Bus Routing: An Application to School Buses in Rural Areas , 2013, Transp. Sci..

[19]  Gilbert Laporte,et al.  Metaheuristics: A bibliography , 1996, Ann. Oper. Res..

[20]  Bharadwaj Veeravalli,et al.  Spatial-temporal traffic speed bands data analysis and prediction , 2017, 2017 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM).

[21]  Frank Neumann,et al.  Optimal Fixed and Adaptive Mutation Rates for the LeadingOnes Problem , 2010, PPSN.

[22]  Reza Tavakkoli-Moghaddam,et al.  Hybrid artificial intelligence and robust optimization for a multi-objective product portfolio problem Case study: The dairy products industry , 2019, Comput. Ind. Eng..

[23]  Kendall E. Nygard,et al.  School bus routing using genetic algorithms , 1992, Defense, Security, and Sensing.

[24]  Silvano Martello,et al.  Meta-Heuristics: Advances and Trends in Local Search Paradigms for Optimization , 2012 .

[25]  Reza Tavakkoli-Moghaddam,et al.  Multiobjective fuzzy mathematical model for a financially constrained closed‐loop supply chain with labor employment , 2019, Comput. Intell..

[26]  Ángel Corberán,et al.  Heuristic solutions to the problem of routing school buses with multiple objectives , 2002, J. Oper. Res. Soc..

[27]  Saman K. Halgamuge,et al.  Particle Swarm Optimization with Self-Adaptive Acceleration Coefficients , 2002, FSKD.

[28]  M. Fisher,et al.  A multiplier adjustment method for the generalized assignment problem , 1986 .

[29]  Hyung Rim Choi,et al.  Development of a Genetic Algorithm for the School Bus Routing Problem , 2015 .

[30]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[31]  Byung-In Kim,et al.  A probability matrix based particle swarm optimization for the capacitated vehicle routing problem , 2012, J. Intell. Manuf..

[32]  Petr Stodola,et al.  Using the Ant Colony Optimization algorithm for the Capacitated Vehicle Routing Problem , 2014, Proceedings of the 16th International Conference on Mechatronics - Mechatronika 2014.

[33]  Byung-In Kim,et al.  The school bus routing problem: A review , 2010, Eur. J. Oper. Res..

[34]  Dulyatat Nualhong,et al.  Selective self-adaptive approach to ant system for solving unit commitment problem , 2006, GECCO '06.

[35]  Randy B Machemehl,et al.  Using a Simulated Annealing Algorithm to Solve the Transit Route Network Design Problem , 2006 .

[36]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[37]  Elizabeth F. Wanner,et al.  A strategy for clustering students minimizing the number of bus stops for solving the school bus routing problem , 2016, NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium.

[38]  Sinem Büyüksaatçi Kiriş,et al.  Metaheuristics Approaches to Solve the Employee Bus Routing Problem With Clustering-Based Bus Stop Selection , 2020 .

[39]  Shaharuddin Salleh,et al.  Simulated annealing technique for routing in a rectangular mesh network , 2014 .

[40]  Erfan Babaee Tirkolaee,et al.  A robust bi-objective multi-trip periodic capacitated arc routing problem for urban waste collection using a multi-objective invasive weed optimization , 2019, Waste management & research : the journal of the International Solid Wastes and Public Cleansing Association, ISWA.

[41]  Xiubin Bruce Wang,et al.  Cluster Based Methodology for Scheduling a University Shuttle System , 2020 .

[42]  Mark Hoogendoorn,et al.  Parameter Control in Evolutionary Algorithms: Trends and Challenges , 2015, IEEE Transactions on Evolutionary Computation.

[43]  Byung-In Kim,et al.  A school bus scheduling problem , 2012, Eur. J. Oper. Res..

[44]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[45]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[46]  Evgeny Burnaev,et al.  Reinforcement Learning for Combinatorial Optimization: A Survey , 2020, ArXiv.

[47]  Maoguo Gong,et al.  Evolutionary Multitasking With Dynamic Resource Allocating Strategy , 2019, IEEE Transactions on Evolutionary Computation.

[48]  Wanqing Li,et al.  Adaptive Ant Colony Optimization Algorithm Based on Information Entropy: Foundation and Application , 2007, Fundam. Informaticae.

[49]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[50]  Warren H. Thomas,et al.  Bus routing in a multi-school system , 1974, Comput. Oper. Res..

[51]  Saeed Yaghoubi,et al.  An Improved Particle Swarm Optimization for a Class of Capacitated Vehicle Routing Problems , 2018 .

[52]  Bharadwaj Veeravalli,et al.  Reinforcement learning-enabled genetic algorithm for school bus scheduling , 2020 .

[53]  Norlida Buniyamin,et al.  Comparative study of Genetic Algorithm and Ant Colony Optimization algorithm performances for robot path planning in global static environments of different complexities , 2009, 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA).

[54]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[55]  Michel Bierlaire,et al.  Decision-Aiding Methodology for the School Bus Routing and Scheduling Problem , 2005, Transp. Sci..