A Deep-Reinforcement-Learning-Based Scheduler for High-Level Synthesis

As the most important stage in high-level synthesis (HLS), scheduling mostly relies on heuristic algorithms due to their speed, flexibility, and scalability. However, designing heuristics easily involves human bias, which makes the scheduling unpredictable in some specific cases. In this paper, we propose a deep-reinforcement-learning (Deep-RL) based scheduler for HLS. It maximumly reduces the human involvement and learns to schedule by itself. Firstly, we introduce a novel state and action representation for constrained scheduling problems, which is the foundation of the learning task. Secondly, we use a training pipeline to train the policy network. Supervised learning is used to initialize the weight of the network, and reinforcement learning is used to improve the performance, which makes the Deep-RL based scheduler practical for HLS. Finally, we compare our scheduler with the ASAP schedule and the optimal ILP schedule. Experimental results show our scheduler can reduce up to 74% resource usage compared with the original ASAP schedule, and the gap between the optimal solution is small. Notably, this is the first work leveraging reinforcement learning in HLS and has great potential to be integrated into different HLS systems.