Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?