New MDP Model and Learning Algorithm for Bus Scheduling Problem with Conditional Signal Priority

Ming Liu (1) , Yecheng Zhao (1) , Feng Chu (2) , Feifeng Zheng (3) , Chengbin Chu (4, 5)

1 Tongji University
2 IBISC - Informatique, BioInformatique, Systèmes Complexes
3 Donghua University [Shanghai]
4 ESIEE Paris
5 COSYS-GRETTIA - Génie des Réseaux de Transport Terrestres et Informatique Avancée

Buses are expected to be punctual, but deviation from the schedule is a common occurrence. Transit signal priority (TSP) and conditional signal priority (CSP) are methods to help the bus perform the schedule better by giving traffic signal priority to the bus. Markov decision process (MDP) is suitable for modeling such sequential decision process. In this article, we point out the Markov property of the system based on the analysis of the bus driving process. We model the process of bus driving with CSP as a Markov decision process. Then, the Deep Q Network (DQN) algorithm is applied to solve this MDP model. To the best of our knowledge, this is the first time that the bus scheduling problem with CSP has been modeled as an MDP model and solved by a learning algorithm. Numerical experiments verify the applicability of the DQN algorithm to solve this MDP model.

Keywords: Smart transportation; Transportation Systems; Decision Support System

Domains: Operations Research [math.OC]

Ming Liu, Yecheng Zhao, Feng Chu, Feifeng Zheng, Chengbin Chu. New MDP Model and Learning Algorithm for Bus Scheduling Problem with Conditional Signal Priority. 10th IFAC Conference on Manufacturing Modelling, Management and Control (MIM 2022), Jun 2022, Nantes, France. pp.3160--3165, ⟨10.1016/j.ifacol.2022.10.215⟩. ⟨hal-03723270⟩