Taming the heavy-tailed features by shrinkage and clipping