Interim report ( COMP 4801 ) AUTONOMOUS DRIFTING RC CAR WITH REINFORCEMENT LEARNING

The advent of self-driving cars has pushed the boundaries on how safe passenger automobiles can be, but most modern self-driving car systems ignore the possibility of a car slipping resulting from inclement weather or driver error. Passengers and bystanders would benefit heavily if self-driving cars could handle slipping by learning to drift with the turn rather than against it (by applying the brakes, or turning away which is the instinctive action), preventing many fatalities [1]. Our project is aimed at studying the drifting (over steering of a car that results in the loss of traction of the rear wheels) of an autonomous remote controlled (RC) car. We use reinforcement learning techniques and algorithms to design a controller for an RC car that learns to drift without human intervention. Reinforcement learning is a branch of machine learning that primarily deals with learning a control agent from trial-and-error, much like how humans learn by interacting with the environment. Reinforcement learning has in recent years been used to learn all sorts of robotic controllers and even defeat the best human player at Go. It is an exciting realm of machine learning, and we decided on using it to teach an RC car to maintain a steady state circular drift. As for the technique employed, we use double dueling deep Q-networks and Q-learning as our primary algorithm. However, using reinforcement learning (RL) typically requires many interactions with the environment before learning anything useful. Since robotic systems are prone to wear with use, we implemented a simulator by modeling the car dynamics, where we run most iterations of the learning algorithm. In addition, since it is imperative to define the reward function appropriately to make sure that our agent learns the right behaviour in the shortest time possible, we also use potential based reward shaping to shape the rewards the agent receives.