Optimizing Gradient-driven Criteria in Network Sparsity: Gradient is All You Need