Higher Gauss sums of modular categories