Double-DQN based path smoothing and tracking control method for robotic vehicle navigation

Abstract Automatically tracking paths with large curvatures smoothly and tightly is an acknowledged challenge in robotic vehicle navigation. This is particularly true for some navigation applications in which pre-defined paths such as crop rows in agricultural applications must be followed accurately. This article presents a novel path smoothing and tracking control method based on Double Deep Q-Network (Double DQN) for an automated robotic vehicle. This Double-DQN-based controller utilized a deep reinforcement learning method that scored all potential actions at the current state according to their performance index and selected the best performer as the output of the network. In this study, a path-tracking algorithm was self-developed with a deep Q-network trained by driving a rover in a simulated virtual environment. The algorithm was tested in both simulation and on a grass field to follow paths with multiple sharp turns. The performance was compared with that of the Pure-Pursuit Control (PPC) algorithm. The results showed that the Double DQN-based control dramatically reduced the settling time and the overshoot at the corner at higher forward speeds at a minor expense of slightly increased rise time and steady state error.