Enhanced data reconciliation of freight rail dispatch data

Abstract In order to enable widespread use of data driven analysis for rail operations problems, large volumes of complete and clean data are needed. In this work a data reconciliation problem for rail dispatch data is proposed to automatically clean and complete noisy and incomplete data. The proposed method finds a minimally-perturbed modification of the observed historical data that satisfies operational constraints, such as feasibility of meet and overtake events. The method is demonstrated on a large historical dataset from freight rail territory in Tennessee, US, containing over 3000 train records over six months. The results show that data reconciliation reduces timing error of imputed points by up to 15% and increases the number of meet and overtake events estimated at the correct historical location from less than 40% to approximately 95%. It is also shown that regularizing the data reconciliation problem with historical train performance data further decreases the error of reconstructed points by 15%, and using an L 2 normalization can reduce mean squared error by over 50%. These findings indicate that the data reconciliation method is a useful preprocessing step for analysis and modeling of railroad operations that are based on real-world physical dispatching data.