Understanding passenger transfer behavior is crucial to designing better multimodal transport networks and improving public transport service quality. However, obtaining data for modeling transfer behavior remains a challenge, in particular under complex facility configurations. The recent emergence of smart card data provides new and efficient data-driven approaches to modeling public transport systems. In this paper, the authors present the effect of time of day, day of week, age of passengers, crowdedness at stop/station and collective pressure in determining passenger transfer time. By analyzing transfer profiles provided by smart card transactions, they apply regression models to assess the impact of each factor in determining passenger behavior under different scenarios. Using morning peak period as a base category, they find that passengers are faster during morning peaks even though it is more crowded. In terms of age of passengers, they find that children and senior citizens generally transfer slower than adults. However, children may outperform adults in passing through an overpass. The crowding effect at a bus stop is not substantial unless passenger demand reaches its capacity, while the crowdedness at fare gantries always delays transferring. Finally, they identify the effect of collective pressure by using the number of fastest/slowest passengers around one individual as a proxy. A fast passenger around one individual is found to reduce his/her transfer time, while a slow passenger may delay the transfer significantly. This work presents some empirical evidence in understanding passenger transfer behavior in a multimodal transit network. The results could be incorporated into physical surveys to better model pedestrian behaviors, supporting convenient facility design and public policy making.