Proactive and defensive driving requires the vehicle drivers, either human beings or software, to predict the intention of surrounding objects so as to prevent accidents. Many trajectory prediction models have been proposed, while most of them are deterministic models, no matter they are numeric approaches or DNN-based approaches. Given the highly dynamic environment, probabilistic reasoning was proved to be effective for robotic control and driving. So are the prediction results. In this work, we proposed a conditional GAN-based trajectory prediction model, which takes into account the observed trajectory, the social behaviors, and physical constraints in the environment, to predict the legitimate and accurate intention of pedestrians, bicyclists, and vehicles. The prediction results enable probabilistic reasoning for driving and robotic control. The experiment results show that our model outperforms the existing deterministic models by reducing 60.35% average displacement error (ADE) and 47.77% final displacement error (FDE) in four seconds prediction, and also outperforms the state-of-the-art non-conditional generative model by reducing 21.74% ADE and 21.34% FDE in four seconds prediction.