Social recommendation has been gaining increasing attention recently with the rapid development of online social network. Existing social recommendation methods are based on the assumption, so-called social-trust, that users' preference or decision is influenced by their social-connected friends' purchase behaviors. However, they assume that the influences of social relation are always the same, which violates the fact that users are likely to share preference on diverse products with different friends. More precisely, friends' behaviors do not necessarily affect a user's preferences and the influence is diverse among difference items even if it exists. In this paper, we contribute a new solution, CSR (short for Characterized Social Regularization) model by designing a universal regularization term for modeling variable social influence. This regularization term capture the finely grained similarity of social-connected friends. We further introduce two variants of applying the term to model optimization with high robustness. Our proposed model can be applied to both explicit and implicit iteration due to its high generality. Extensive experiments on two real-world datasets demonstrate that our CSR significantly outperforms state-of-the-art social recommendation methods. Further experiments show that CSR can effectively improve recommendation performance for those users with sparse social relations or behavioral interactions.