Depriving the Survival Space of Adversaries Against Poisoned Gradients in Federated Learning