Effect of glassy topological disorder on low-energy magnetic excitations