Four-dimensional topological lattices without gauge fields