Symmetries and topological operators, on average