In 2018, around 22,000 people were killed in road accidents in Thailand and more than 80% of road accidents involved motorcycles. There are several reasons and one of them is motorcycles driving in the wrong direction. Our paper proposes a system for road surveillance to detect and track moving vehicle's direction on two-way traffic roads from CCTV viewpoint using deep learning where the roadside noise is filtered out. The system can determine whether the vehicles drive in the correct direction in their lane. In this case, we will focus on motorcycles. To set up the system, there are two main steps: draw the area of interest and validate the correct direction. In the direction validation process, we detect and track vehicles by using YOLOv4-tiny and Deep SORT tracking algorithm. The system returns a satisfactory and accurate result for detecting direction without noise, such as moving objects on the sidewalk or moving vehicles in an unwanted area.