Randomized benchmarking for qudit Clifford gates