On consistent weakly-nonlocal modeling of granular media