Road passage time is an important measure of urban traffic. Accurate estimation of road passage time contributes to the route programming and the urban traffic planning. Currently, the estimation of road passage time for a particular road is usually based on its historical data which is simple to express the general law of road traffic. However, with the increase of the number of roads in the urban area, the connection between roads becomes more complex. The existing methods fail to make use of the connection between different roads and the road passage time, merely based on its own historical data. In this paper, we propose a road passage time estimating model, called “CA-RPT”, which utilizes the contextual information between road connections as well as the date and time period. We evaluate our method based on a real geolocation information data set collected by mobile APP anonymously. The results demonstrate that our method is more accurate than the state-of-the-art methods.